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The California School Psychologist 
as a Quintessential Resource 

Shane R. Jimerson 

University of California , Santa Barbara 



The California School Psychologist is a quintessential resource, providing valuable information 
to promote the academic success and facilitate the developmental trajectories of diverse students. 
Articles published in The California School Psychologist enhance knowledge, science, and practice 
related to school psychology. Each year over 3,000 volumes of The California School Psychologist 
journal are distributed directly to school psychologists and other educational professionals. In addition, 
many professionals seek out manuscripts published in The California School Psychologist upon revealing 
articles of interest in searches of international electronic literature databases (e.g .,ERIC, developed by 
the US Department of Education and PsycINFO , developed by the American Psychological Association). 
Other professionals access the contents of The California School Psychologist on the internet at 
www.education.ucsb.edu/school-psychology. Thus, educational professionals throughout the state, 
across the country, and around the world utilize information published in The California School 
Psychologist 

Articles in this volume provide valuable information addressing an assortment of important issues 
in the field, including: the translation and validation of four nonverbal subtests of the Differential 
Abilities Scales (DAS) for use with Spanish-speaking students; applied research examining family- 
centered practices in an ethnically diverse elementary school; consultation, collaboration, and support 
for new teachers; a longitudinal study providing further insights regarding the association between 
grade retention and school dropout; a synthesis of best practices in assessing kindergarten readiness; 
and the use of a formative program portfolio process to enhance graduate school psychology training. 
The following highlights from each article provide an overview of the topics addressed in this volume. 

The first article (Sandoval, Antunez-Bellatin, & Lewis, 2002) provides a summaiy of the transla- 
tion and validation of the (DAS) nonverbal scales for use with Spanish-speaking children. The exi- 
gency of appropriate assessments of cognitive skills for Spanish-speaking children is increasing as the 
number of individuals of Latino descent has grown by sixty percent nationally. Moreover, in states 
such as California and Texas, Latino youth are emerging as the majority, having grown sixteen percent 
faster than any other group during the past decade. This article begins by describing procedures that 
may be used to translate directions of a test from one language to another. This study included admin- 
istration of the Spanish translation version of the four nonverbal DAS subtests to a referred population 
of 97 Spanish-speaking children. In addition, students were assessed with the Test of Nonverbal Intel- 
ligence (TONI), the Comprehensive Test of Nonverbal Intelligence (CTONI), and the Woodcock-Munoz 
Language Survey The analyses yielded preliminary evidence of internal validity, concurrent validity, 
and construct validity. The authors suggest that the Spanish translation of the DAS nonverbal subtests 
may provide a more standard administration of this assessment to Spanish-speaking students. Finally, 
the authors encourage further research exploring the Spanish translation of the DAS nonverbal subtests. 
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The second article (Ho, Robinette, & Gonzales, 2002) examines family-centered practices in an 
ethnically diverse elementary school. Considering the current context of education policies advocat- 
ing family-centered practices, this applied research study explored families’ and educators’ percep- 
tions of family-centered practices. This study included twelve educational professionals and 1 29 families 
who completed surveys examining “typical” and “ideal” family-centered practices. Exploratory fac- 
tor analyses yielded three main factors: Positive Relating with Families, Partnering with Families, and 
Family-Focused Approaches. Moreover, analyses indicated no significant differences between fami- 
lies and educators for either “typical” or “ideal” ratings of family-centered practices, with both desir- 
ing a higher level of family-centered practices. While single-parent families rated family-centered 
practices lower than two-parent families, there were no differences among families by ethnicity or 
home language. This article emphasizes the importance of linkages between family support systems, 
acceptance of family differences, and the identification and use of strengths in families to facilitate the 
success of children at school. The authors encourage school psychologists to provide leadership in 
promoting family-centered practices in schools. 

The third article (Knotek, Babinksi, & Rogers, 2002) highlights elements of a teacher induction 
program, emphasizing mentoring, reflective practice, and collaboration. This qualitative study exam- 
ined how the process of consultee-centered case consultation (CCC) facilitated collaboration, prob- 
lem-solving, and professional development in a new teacher group (NTG). It was found that partici- 
pants of the new teacher group benefited in terms of more positive self-perception and an enhanced 
approach to problem solving. Amidst the context of teacher shortages in many states, many districts 
are receptive to innovative proposals that will contribute to professional development and support 
teacher retention. School psychologists can offer developing professionals a supportive, dynamic 
forum in which teachers are encouraged to reflect on their practices, to explore problems and solu- 
tions, and are supported to continue their professional development and meet the needs of children. In 
addition, through the process of engaging new teachers in a positive consultation experience during 
the formation of their careers, school psychologists have the opportunity to broaden their base of peers 
who value and make -use of indirect services. 

The fourth article (Jimerson, Ferguson, Whipple, Anderson, & Dalton, 2002) provides further 
information regarding the association between grade retention and high school dropout. This prospec- 
tive longitudinal study examines within-group differences, exploring the characteristics of those stu- 
dents who are retained and subsequently drop out as compared to those students who are retained and 
do not drop out. The authors invoke a transactional-ecological developmental framework to assist 
interpretation of the findings within the context of long-term outcomes across development. Results 
reported in this study indicated that there are early socio-emotional and behavioral characteristics that 
distinguish those retained students who are most likely to drop out of high school. Furthermore, 
maternal level of education and academic achievement in the secondary grades were also associated 
with high school graduation status. The authors suggest that it is especially important to attend to the 
socio-emotional and behavioral adjustment of children throughout their schooling to facilitate both 
immediate and long-term academic success. 

The fifth article (Pyle, 2002) explores best practices in examining kindergarten readiness. Em- 
phasizing the first national educational goal, “All children in America will start school ready to learn,” 
this article addresses related challenges and delineates important considerations and strategies in as- 
sessing kindergarten readiness. This article reviews the history, related research, and methodological 
concerns in examining definitional and measurement aspects of screening instruments for school readi- 
ness. The author also includes a discussion of important psychometric properties such as: predictive 
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validity, sensitivity and specificity, reliability, test floor, and item gradient. Based on a review of 
extant literature, four suggestions are offered to enhance readiness screening assessment: (a) define the 
purpose of the assessment, (b) select ecologically focused instruments with multiple raters and follow- 
up procedures, (c) determine the process for conducting the assessment, and (d) think carefully about 
how to analyze, interpret, and use the results. 

The sixth article (Hass & Osborn, 2002) examines the implementation of a formative program 
portfolio in a graduate program in school psychology. The portfolio process described in this article 
emphasizes Donald Schon’s work regarding the reflective practitioner and the notion that professional 
knowledge and practice is enhanced by making what is tacit, implicit. The article includes: a descrip- 
tion of the connections with training standards, program course work, exit interview questions, and 
student reflections on the portfolio process. The program portfolio described in this article incorpo- 
rates the training standards established by the National Association of School Psychologists (NASP). 
The authors report that the portfolio process has served to strengthen the graduate program and con- 
tributes to developing students’ higher level thinking skills and critical evaluation. The authors also 
propose that the portfolio process has benefits beyond graduate training and could be valuable for 
those already in the field. 

Considering the scope of professional preparation in school psychology (e.g., child development, 
learning theory, psychology, prevention and intervention program planning and evaluation, consulta- 
tion and collaboration, assessment, socio-emotional development, sociocultural considerations, pro- 
fessional leadership, human relations, school safety and violence prevention, wellness promotion, cri- 
sis intervention, counseling, family-school collaboration, research, measurement, technology, educa- 
tional law, professional ethics, supervision, and mentoring), school psychologists are invaluable in 
promoting the social and cognitive competence of students. As reflected by the series of articles in this 
volume emphasizing a breadth of knowledge, (i.e., the assessment of diverse students, family-cen- 
tered practices at school, consultation and collaboration with new teachers, the association between 
early grade retention and subsequent outcomes, assessment of kindergarten readiness, and using for- 
mative portfolio assessment to enhance the preparation of school psychologists) The California School 
Psychologist is a quintessential resource in facilitating the education and development of children. 
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Using the DAS Nonverbal Scales with Spanish-Speaking 
Children: Translation and Validation 



Jonathan Sandoval 
University of California, Davis 

Malu Antunez-Bellatin and Sharon Lewis 
Lodi, California School District 



This paper describes the procedures that can be used by school professionals to translate the 
directions of a test from one language (English) to another (Spanish). The procedures were then 
applied to the translation of four nonverbal Differential Ability Scale subtests into Spanish. The 
translation of these directions is provided in an appendix. The performance of a referred popula- 
tion of 97 Spanish-speaking children, who were given the translated directions, mirrors that of 
children in the DAS standardization sample. Evidence from internal validity coefficients, concur- 
rent validity coefficients, and construct validity suggests that the DAS nonverbal subtests can be 
translated and yield information comparable to non-translation. The translation provided makes 
possible a more standard administration of this test to Spanish-speaking children. 

Keywords: Differential Ability Scales, DAS, Spanish, Translation, Validation 



Perhaps the first option that comes to mind to psychologists as well as others (such as the courts) 
when faced with a non- or limited-English proficient speaker is to have the test of interest translated 
into the language of the test taker. Translating tests in situ or beforehand by a bilingual psychologist 
and especially the use of a non-psychologist translator in testing are fraught with hazards and issues. 
Translating a test from one language to another, and at the same time preserving the content, difficulty 
level, reliability, and validity is a daunting undertaking. It is rarely done successfully at the local level, 
because word frequencies (difficulties) and verbal concepts are difficult to match across languages. 
This difficulty is particularly acute with respect to the verbal content of the test for several reasons: (a) 
word frequency, difficulty and meaning are seldom exactly the same across languages; (b) language is 
a part of a culture and concepts have cultural nuances; (c) there are likely to be social class differences 
in those who do the translation and those who take the test; (d) information may be lost because of 
nonverbal communication and context in the original; and (e) content and tasks may have different 
levels of practice and exposure. Because of these problems, it is best not to translate verbal tests at the 
local district level. 

With nonverbal content, translation must alter the directions given the child. It is much easier to 
translate instructions and procedures than it is to translate test items or questions. However, this task 
also should not be undertaken lightly for some of the reasons mentioned above. In addition, once a 
translation has been produced it still must be examined for reliability and validity. 

In assessing the intellectual functioning of English Proficient children, the authors have been 
impressed with the utility of the Differential Abilities Scales (DAS; Elliott, 1990a). This test yields 
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useful information about children, who find it engaging and not too taxing, and the results have helped 
us to plan successful interventions for children. It has construct validity across Hispanic, Black, and 
White children (Keith, Quirk, Schartzer, & Elliott, 1999). In addition, the DAS author (Elliott, 1990a, 
p. 36) suggests that the Special Nonverbal Composite (SNC) scale may be used with children who are 
not proficient in English speaking. Riccio, Ross, Boan, Jemison, and Houston (1997) found that the 
SNC scores obtained by 10 children ages 3 to 6, who had English as a second language or were limited 
English proficient, were comparable with a comparison group of children who had English as their 
primary language. The authors do not indicate if the test was translated. 

The Nonverbal Scale is made up of four subtests that use a minimum of verbal directions and 
employ symbolic stimuli. Recall of Designs requires a child to draw a non-pictorial line drawing from 
memory. Pattern Construction is a block design, visual pattern test. Sequential and Quantitative Rea- 
soning requires a child to identify the missing part of a series of figures or numbers. Matrices involves 
supplying the missing element from a pattern displayed in a matrix. Although Elliott recommends the 
use of these four subtests with non-English-speaking children, he only states that the instructions can 
be communicated in the child’s primary language or by gesture. No translations of the directions to the 
nonverbal subtests have been provided in the manual or elsewhere. Given the absence of published 
non-English directions for the DAS Nonverbal subtests, there is a need for other language directions to 
create a standardized administration. Spanish directions are a priority because this language is so 
common in U.S. and California schools. 

Prior to beginning a translation, best practices should be considered. Geisinger (1 994) provides a 
step-by-step approach that can be used in translating test directions: 

1 . First the directions must be translated and adapted to the new language . In some instances, this 
can be done more or less in a word-by-word or sentence-by-sentence fashion, but in others the con- 
cepts are translated on a conceptual level. The usual approach is to use back translation. In back trans- 
lation, first one translator converts the information into the target language, and then a second indepen- 
dently translates the information back into English. Both translators must be fluent in both English and 
in the target language. Judges next compare the version translated back to the original. This process 
can be repeated to insure accuracy. It is best that the translators be unaware that their translation will be 
subject to back translation. If they are unaware of the reason for the translation, they will be less likely 
to choose wording that will back translate than to choose words that will capture best the meaning of 
the original. A flaw in the back translation process is that there may be too much emphasis on having 
the original language returned by the second translator so that a stilted and non-optimal translation 
may emerge. 

2. Next, a panel of expert bilingual and bicultural individuals serving as a focus group reviews 
the new version of the translated or adapted directions. Geisinger ( 1 994) suggests that the panel mem- 
bers, “(a) review the items and react in writing, (b) share their comments with one another, and (c) 
meet to consider the points made by each other and to reconcile any differences of opinion” (p. 306). 

3. After the panel has deliberated without the original translator present, the translator further 
adapts the draft instructions on the basis of the reviewers ’ comments . 

4. In the next step, the directions are given a pilot trial with a small group of typical examinees. 
Afterward, the test takers are interviewed to determine if they understood the instructions. The transla- 
tion should be modified in response to the findings from these interviews. 

5. At this point, the translated directions are ready for a field test. It should be administered to a 
large enough representative sample to yield data to examine issues related to the reliability and validity 
of the test. 
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Adapting a set of directions for a standardized test is a serious undertaking. The purpose of this 
study was to create a Spanish translation of the directions to the DAS Nonverbal scale subtest, and to 
collect data concerning the validity of this translation. We sought to learn if referred children tested 
with this translation show the same patterns of scoring and correlations with other tests as did the 
children in the standardization sample. 

METHOD 



Participants 

Professionals . Twelve credentialed professionals, nine school psychologists and three speech and 
language specialists, participated in the initial translation of the directions into Spanish. All were na- 
tive language speakers or competent second-language speakers with extensive experience assessing 
Spanish-speaking children. Four different non-psychologist professional translators, experienced and 
skilled in both Spanish-to-English and English-to-Spanish interpretation, completed the back transla- 
tions into English. The original 12 professionals worked as a group to resolve any wording issues, to 
try out the translations with children to determine their comprehension, and to produce the final Span- 
ish translation. 

Children. For the validation phase of the study, 97 children were assessed. Their language compe- 
tence was determined by the Language Assessment Scales (DeAvila & Duncan, 1990). The results 
indicated that all were primarily Spanish speakers with Spanish as the language spoken in the home. 
Their ages ranged from 6 years, 0 months to 14 years, 4 months with a median of 8 years, 8 months. 
Fifty-three of the children were tested as an initial special education referral, 20 of the children were 
referred for language screening, 1 0 were volunteered by their parents, four were reevaluated for place- 
ment in special education, five were referred for testing as potential participants in the Gifted And 
Talented Education program, and five were general referrals for screening. The children came from 
four moderately sized suburban/rural school districts in the Sacramento/San Joaquin Valley of North- 
ern California. 

Measures 

The DAS is an American adaptation of the British Abilities Scales (Elliott, 1990a). The Matrices 
and Sequential and Quantitative Reasoning subtests together yield a cluster score Nonverbal Reason- 
ing Ability. The Recall of Designs and Pattern Construction subtests yield a cluster score Spatial Abil- 
ity. All four tests together yield a Special Nonverbal Composite. 

Two commonly used nonverbal tests were used to cross-validate the Spanish translation. The Test 
of Nonverbal Intelligence (TONI) is a language-free test using abstract symbols and figures yielding a 
single score (Brown, Sherbenou, & Johnsen, 1997). The Comprehensive Test of Nonverbal Intelli- 
gence (CTONI) contains six subtests (Pictorial Analogies, Geometric Analogies, Pictorial Categories, 
Geometric Categories, Pictorial Sequences, and Geometric Sequences), which are combined to yield 
an overall Nonverbal Intelligence Quotient (Hammill, Pearson, & Wiederholt, 1997). 

The Woodcock-Munoz Language Survey is a screening instrument used to establish an individual’s 
language proficiency in English and Spanish (Woodcock & Munoz- Sandoval, 2001). It contains mea- 
sures of Oral Language (two tests: Picture Vocabulary and Verbal Analogies), reading and writing. 
Parallel tests are available in Spanish and in English. For this study, the English and Spanish Oral 
Language tests were administered. 
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Procedure 

The directions to the four DAS subtests were translated in accordance to the strategy outlined by 
Geisinger. The second and third authors initially translated the directions into Spanish, the second 
author’s first language and a second language for the third author. This translation was discussed and 
modified by nine other professional colleagues. When a consensus translation was achieved, four fully 
Spanish-English bilingual non-psychologists back-translated the directions into English. After verify- 
ing the accuracy of the translation (Step l), the translation was offered to the group of Spanish bilin- 
gual' school psychologists and speech therapists, who, serving as a focus group, examined the back 
translation and revised the Spanish translation (Step 2). The results of the focus group feedback sug- 
gested only minor changes, which were made by the second and third authors (Step 3). The resulting 
translation was piloted with the next three referrals to the authors and found to be comprehensible 
(Step 4). Finally the directions were field tested with the next 97 children Spanish speaking children 
referred to the second and third authors (Step 5). 

RESULTS 



Translation 

The appendix contains the resulting final Spanish translation. The English directions may be found 
in the DAS administration manual. 

Validation 

Table 1 lists the mean and standard deviation of the DAS scores for the total sample and for four 
subsamples of referred children. The means on the tests and composite scores for the total group are 
about one-half to two-thirds of a standard deviation below the DAS norm group average. Disaggregat- 
ing the children by reason for referral, the special education referrals and the language referrals scored 
about two-thirds of a standard deviation below the mean, but the gifted referrals were one and a half 
standard deviations above the mean, and the volunteers were above average. 

Table 2 presents the intercorrelations of the four DAS tests and their correlations with the TONI 
and CTONI. Intercorrelations from the DAS manual Table 9.3 are included in brackets (Elliott, 1990b). 
The obtained intercorrelations in Table 2 are very close to those reported in the DAS manual in three of 
the cases, are within .10 points in two cases, and higher in one case. The DAS score with the highest 
correlation with other tests is the Matrices test, which correlates most highly with the CTONI total 
score. All of the tests correlate significantly with the TONI. Two of the DAS subtests, Recall of De- 
signs and Sequential and Quantitative Reasoning, do not correlate significantly with the CTONI total. 

Table 3 contains the correlations between the DAS composite scores and the TONI and CTONI. 
The correlations are significant but are slightly below the correlations of the DAS scores and the 
WISC-R Performance IQ scores reported in the DAS technical manual. The correlations of the Special 
Nonverbal Composite with the TONI and CTONI are .60 compared to a correlation of .71 reported for 
the WISC-R (Table 9.26 DAS technical manual). 

The DAS composite intercorrelations are almost identical to those reported in the DAS Technical 
manual. For this sample of special education referred Spanish-speaking children, the DAS scores 
obtained using the created Spanish directions appear to perform the same way as they did in the stan- 
dardization sample. 

The data displayed in Table 4 show the correlations between the DAS composites and four scores 
from the Woodcock-Munoz battery: English Oral Language, Spanish Oral Language, Spanish Picture 
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Table 1 

Mean and Standard Deviation for the Total Group and for Four Types of Referrals . 
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Vocabulary (Vocabulario sobre dibujos) and Spanish Verbal Analogies (Anologias verbales). All of the 
correlations are significant. There is a moderate correlation with language skills in both languages, but 
a relatively small correlation with Spanish picture vocabulary. The strong correlation with Verbal Analo- 
gies, the most cognitive of the Spanish language tests, provides evidence for concurrent validity of the 
Special Nonverbal Composite obtained with the translated directions. 

DISCUSSION 

Following Geisinger’s (1994) procedures for test translation, we produced a Spanish set of direc- 
tions for the DAS nonverbal tests. The procedure, albeit time consuming, resulted in a translation that 
a group of professionals found to be accurate and easily understood by children. 

Evidence for the differential group validity of the translated test directions is the differential and 
expected performance of various groups of referred children. Although only a small number of chil- 
dren, those thought potentially gifted performed well on the test. A larger number of children referred 
for special education by their teachers performed below average on the DAS tests and composites. 



Table 2 

Intercorrelation of DAS Tests and Correlation of DAS Tests with TONI and CTONL 





Pattern 

Construction 


Recall 
of Designs 


Matrices 


Sequential & 
Quantitative Reasoning 


Recall of Designs 
(n = 97) 


.47 [.57] 








Matrices 
(n = 97) 


.54 [.53] 


.65 [.44] 






Sequential & 
Quantitative Reasoning 
(" =97) 


.44 [.54] 


.44 [.44] 


.58 [.58] 




TONI 
{n = 60) 


.47 


.43 


.59 


.52 


CTONI 
(« = 34) 


.55 


.28 


.71 


.24 



Note. All correlations significant beyond p < .01 except those for CTONI and Recall of Designs and 
Sequential and Quantitative Reasoning. Intercorrelations from the DAS manual Table 9.3 are 
included in brackets. 



TONI = Test of Nonverbal Intelligence 

CTONI = Comprehensive Test of Nonverbal Intelligence 
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Table 3 

DAS Cluster Scores Correlations with the TONI and CTONI and Cluster Score Intercorrelations, 





TONI 


CTONI 


Spatial 

Reasoning 


Special 

Nonverbal 


Composite 

Nonverbal 


.61 


.52 


.66 [.61] 


.91 [.90] 


Reasoning 


(n = 60) 


(n = 34) 


(n = 97) 


(« = 97) 


Spatial 


.52 


.50 




.91 [.90] 


Reasoning 


(n - 60) 


1? 

II 

U) 

4^ 




(* = 97) 


Special 

Nonverbal 

Composite 


.61 

(n = 60) 


.59 

(n = 34) 







Note. All correlations significant beyond p < .01. Intercorrelations from the DAS technical manual 
Table 9.3 are included in brackets. 



TONI = Test of Nonverbal Intelligence 

CTONI = Comprehensive Test of Nonverbal Intelligence 



Table 4 

DAS Scores and Oral Language Measures, Picture Vocabulary and Verbal Analogies from Woodcock - 
Munoz Language Survey. 





Nonverbal 

Reasoning 


Spatial 

Reasoning 


Special Nonverbal 
Composite 


English Oral 


.55** 


5 1 ** 


.56** 


Language 


(n = 73) 


(* = 73) 


(* = 73) 


Spanish Oral 


.45** 


.45** 


.48** 


Language 


(* = 79) 


(* = 79) 


(* = 79) 


Spanish Picture 


.24* 


.32** 


.30** 


Vocabulary 


(* = 76) 


(* = 76) 


(* = 76) 


Spanish Verbal 


.62** 


.64** 


.68** 


Analogies 


(* = 76) 


(n = 76) 


(* = 76) 


Note. **p<.01, 


* p < .05) 
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Evidence of the concurrent validity is the expected correlation of the tests and composites with 
other tests of mental ability, both other nonverbal tests and a Spanish language test. Evidence for the 
internal validity of the test is the same degree of intercorrelation in this sample as in the standardiza- 
tion sample. The DAS nonverbal tests, using Spanish directions, appear to yield scores that correlate 
with other measures the same way as the tests do using English directions. We believe this Spanish 
translation shows promise and permits the nonverbal tests of the DAS to be used in a standardized 
fashion with Spanish-speaking children. 

For school psychologists who use the DAS Spanish directions, it may be useful to give instruc- 
tions in both languages spoken by the examinee. This practice will give the examinee an optimal 
chance to understand what is expected. This option will be particularly valid for children who are 
English learners and are used to such procedures in bilingual classrooms. Strictly speaking, a dual 
language version of a test should also be standardized and normed with bilingual and monolingual 
populations. Increasingly this is being done with bilinguals included in the norm sample, as was done 
with the Woodcock-Johnson III. 

It would also be useful to examine item performance data to establish that the test items perform 
the same way with this Spanish-speaking sample as with the standardization sample. This examination 
of test fairness was beyond the scope of this study, but is a logical next step. This study is limited 
because the children studied were a referred population, and thus not representative of the entire range 
of abilities. Nevertheless the results thus far suggest that this translation may standardize administra- 
tion of the DAS SNC for Spanish-speaking children. 
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Appendix 

Spanish Translation of the four DAS subtests making up the Special Nonverbal Composite 

Memoria de Dibujos 

Materiales: 

Booklet 1 

Lapiz con borrador 

Hojas de papel en bianco de aproximadamente 4 por 5 pulgadas 
Cronometro 

Comienzo: Todas las edades 
Ejemplo A 

Senale el dibujo y diga: 

Aqui hay un dibujo. Miralo con atencion y trata de recordarlo. 

Comience a tomar tiempo. Pasados 5 segundos, retire el estimulo, senale la hoja de respuestas y 
diga: 

Ahora dibujalo aqui. Hazlo igual al que acabas de ver. 

Cuando el nino complete el dibujo, abra el libro en el ejemplo A. Coloque el dibujo estimulo junto 
al dibujo del nino y diga: 

Veamos cuan bien lo haz dibujado. 

Eiemplos B v C 

Retire la hoja de respuesta del item anterior y de una nueva hoja de papel al nino diciendo: 

Ahora mira el siguiente dibujo. Miralo con atencion y trata de recordarlo. 

Presente el siguiente dibujo-estimulo por 5 segundos. Retire el estimulo y senalando la hoja de 
respuesta diga: 

Ahora dibujalo aqui. 

Cuando el nino complete su dibujo, coloque el dibujo-estimulo junto al dibujo del nino y 
comparelos, como en el ejemplo A. Haga comentarios solo en los ejemplos A, B y C. No haga 
ningun comentario en los siguientes dibujos. 

Itemes 1-21 

Itemes 1-21 se administran todos de la misma manera. Para cada uno, coloque primero la hoja de 
respuesta numerada delante del nino y diga: 

Ahora mira este. 

Muestre el item por 5 segundos, retire el estimulo y diga al nino: 

Ahora dibujalo aqui. 
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(1) Punto de Decision - edades: 5.0-7.11 (despues de administrar item 12) 

(2) Punto de Decision - edades: 8.0-11.11 (despues de administrar item 16) 

(3) Punto de Decision - edades: 12.0 - 17.1 1 (despues de administrar item 21) 

Si el nino falla en Itemes 16-17 (6 24-25) (6 en item 31), instruyalo ensenandole como en el ejemplo 
E. En estos itemes, reconozca las respuestas correctas del nino. 

(4) Punto de Decision - edades: 7.0 - 10. 1 1 (despues de administrar item 23) 

(5) Punto de Decision - edades: 1 1 .0 - 14, 1 1 (despues de administrar item 30) 

(6) Punto de Decision - edades: 15.0 - 17.1 1 (despues de administrar item 39) 

Construction de Pair ones 



Materiales: 

Set A (Itemes 1-7) 

Seis cuadrados de caucho esponjoso de color amarillo-y-negro 
Booklet 2 

Set B (Itemes 8-23) 

Nueve bloques deplastico de color amarillo-y-negro 

Cronometro 

Como tomar el tiempo 

(1) Comience a tomar el tiempo inmediatamente despues de dar las ins truce iones. Anote el tiempo 
que el nino demora en completar el patron. 

(2) El tomar el tiempo de las respuestas no se le explica al nino hasta el item 8. Si el 
Nino pregunta sobre el cronometro antes del item 8, diga: 

Te voy a tomar el tiempo con este reloj , pero trabaja con cuidado y trata de hacer bien estos 
patrones. 

Comienzo: Edades 3. 0-6 A 1 
Ejemplo A 

Modelo 

Segundo Ensayo 

Tiempo Limite: Aproximadamente 30 segundos por ensayo 

Coloque dos cuadrados frente al nino, uno con el lado amarillo y el otro con el lado negro hacia 
arriba. Tome dos cuadrados para su propio uso y diga: 

Cada una de estas piezas tiene un lado amarillo y un lado negro. Mira tus piezas por 
los dos lados. 
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Haga una pausa mientras el nino y usted voltean sus piezas. Luego diga: 

Las podemos poner juntas para formar un patron. Mirame ponerlas juntas. 

Haga un modelo con sus piezas, con el cuadrado negro a la izquierda del nino y el cuadrado amarillo 
a la derecha del nino. Deje el modelo delante del nino y diga: 

Ahora pon tus piezas juntas, para que se vean exactamente como las mias. 

Segundo Ensayo: Si la respuesta del nino es incorrecta . o el nino no completa el patron en 30 
segundos, siga el procedimiento del 2do. Ensayo: Senale el error. Haga una demostracion y diga: 
Trata de nuevo. 

Item 1 

Modelo y figura 

Segundo Ensayo 

Tiempo Li mite: Aproximadamente 30 segundos por ensayo 

Retire los cuadrados del nino y el modelo del Ejemplo A. Abra el Booklet 2 en el Item 1 y diga: 
Ahora vamos a hacer el patron que esta en esta figura. Mira. 

Use dos cuadrados para hacer el modelo del Item 1 junto a la figura. Dejando el modelo y la figura 
delante del nino diga: 

Ahora tu haz el mismo patron con tus piezas. 

Segundo Ensayo: (Si es necesario) Senale el error. Haga una demostracion. Diga: 

Trata de nuevo. 

De 3 puntos para un patron correcto en el primer ensayo, 1 punto para un patron correcto en el 
segundo ensayo, y 0 puntos si el nino responde de otro modo. 

Ejemplo B 

Figura 

Segundo Ensayo 

Tiempo Limite: aproximadamente 30 segundos por ensayo. 

Desarme el modelo del Item 1 y ponga los cuadrados a un lado. Mexcle los cuadrados del nino. 
Prepare su cronometro. Pase al Ejemplo B y diga: 

Ahora trata de hacer este. Comienza. 

Comience a tomar tiempo. 

Segundo Ensavo: (si es necesario) Senale el error. Demuestre. 



Trata de nuevo. 
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Item 2 

Figura y Demostracion 

Segundo Ensayo 

Tiempo Limite: aproximadamente 30 segundos por ensayo. 

Mezcle los cuadrados del ejemplo B y de al nino 2 cuadrados mas (un total de cuatro cuadrados). 
Pase al Item 2 y diga: 

Este patron es mas grande. Mi'rame. 

Usando los cuadrados del nino, haga una demostracion del patron. Senale la figura y los cuadrados y 
diga: 

Ves? Son iguales. 

Mezcle los cuadrados del nino y senalando la figura diga: 

Ahora pon tus piezas juntas para que se vean igual que este. 

Comience a tomar el tiempo. Si el nino no usa todas las piezas diga: 

Usa todas las piezas. 

Segundo Ensayo: (si es necesario) Senale el error. Demuestre. 

Trata de nuevo. 

Item 3 

Figura 

Tiempo Limite: 45 segundos 

Ahora trata de hacer este. 

Comience a tomar el tiempo 

Item 4 

Figura y Demostracion 

Segundo Ensayo 

Tiempo Limite: 45 segundos por ensayo. 

De al nino dos cuadrados mas (un total de 6). Pase al Item 4, senale el patron y diga: 

Ahora vamos a usar todas estas piezass para hacer este patrdn. Mirame. 

Construya el patron usando los cuadrados del nino. Luego mezcle los cuadrados y diga: 

Ahora trata tu. 

Comience a tomar el tiempo. 

Segundo Ensayo: (si es necesario) Senale el error. Demuestre. 

Trata de nuevo. 
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Itemes 5-7 

Figura 

Tiempo Limite: 60 segundos por item. 

Mezcle los cuadrados del nino, pase a la pagina correspondiente y diga: 

Ahora trata de hacer este. 

Comience a tomar el tiempo. 

Despues del item 7, retire los cuadrados y el Booklet 2. 

(7) Punto de Decision - Edades 3.0-4. 1 1 (despues de administrar el item 7) 

Eiemplo C 

Modelo y Figura 
Segundo Ensayo 

Tiempo Limite: 30 segundos por ensayo. 

Tenga listos el Booklet 1 y cuatro bloques de plastico. Coloque dos bloques delante del nino y diga: 
Mira estos bloques. Tienen diferentes lados. 

Voltee los bloques para mostrar los diferentes lados. Diga: 

Mira tus bloques por todos los lados. Todos los bloques son iguales. Los podemos poner juntos 
de modo que las partes de arriba formen este patron. 

Abra el Booklet 1 en el Ejemplo C y diga: 

Mira como lo hago. 

Construya el modelo al lado del Booklet 1 de modo que el nino los pueda comparar. Deje el modelo 
en su lugar. De dos bloques mas al nino y diga: 

Ahora hazlo tu. Avisame cuando hayas terminado. 

No tome el tiempo en este ensayo. 

Segundo Ensayo 

Si el nino responde incorrectamente o si el nino no completa el patron correctamente pasados 30 
segundos, siga el procedimiento del Segundo Ensayo: Senale el error, demuestre y diga: 

Trata de nuevo. 

Item 8 
Figura 

Segundo Ensayo 

Tiempo Limite: 30 segundos por ensayo. 

Retire el modelo del item 7. Mezcle los bloques del nino, pase al item 8 y diga: 
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Ahora trata de hacer este. Te voy a tomar el tiempo con este reloj, pero trabaja con cuidado. 
Avisame cuando hayas terminado. 

Comience a tomar el tiempo. 

Segundo Ensavo : (si es necesario). Muestre el error y diga: 

Trata de nuevo. 

Item 9 
Figura 

Segundo Ensayo 

Tiempo Limite: 30 segundos por ensayo 
Mezcle los bloques del nino, pase al item 9 y diga: 

Ahora trata de hacer este. 

Comience a tomar el tiempo. 

Segundo Ensayo (si es necesario). Muestre el error y diga: 

Trata de nuevo. 

Item 10 
Figura 

Tiempo Limite: 30 segundos 

Mezcle los bloques del nino, pase al item 10 y diga: 

Ahora trata de hacer 6ste. 

Comience a tomar el tiempo. 

(8) Punto de Decision - edades 5.0-6. 1 1 (despues de administrar el item 1 0) 

Itemes 11-13 

Figura 

Tiempo Limite: 30 segundos (item 11) 

40 segundos (itemes 1 2 & 13) 

Mezcle los bloques del nino, pase al item 1 0 y diga: 

Ahora trata de hacer este. 

Comience a tomar el tiempo. 

Comienzo: Edades 13.0-17,11 
Ejemplo D 

Figura y Demostracion 

Segundo Ensayo 
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Tiempo Limite: 60 segundos por ensayo 

De al nino dos bloques mis (un total de cuatro) y diga: 

Aqui hay cuatro bloques. Vas a necesitarlos todos para hacer este patron. Mira como lo hago. 

Usando los bloques del nino, construya el patron y coloque el Booklet 1 junto a los bloques para que 
el nino pueda comparar los patrones. Luego mezcle los bloques del nino, deselos y diga: 

Ahora -hazlo tu. Avisame cuando hay as terminado. 

Se gundo Ensayo: Si el nino responde incorrectamente o no completa el patron en 60 segundos, siga 
el procedimiento del Segundo-Ensayo. Senale el error. Demuestre. Diga: 

Trata de nuevo. 

Itemes 14-15 

Figura 

Segundo Ensayo 

Tiempo Limite: 60 segundos (Item 14) 

90 segundos (Item 1 5) 

Mezcle los bloques del nino y diga: 

Ahora trata de hacer este. 

Comience a tomar el tiempo. 

Segundo Ensavo : (sifuese necesario) Senale el error. Demuestre. 

Trata de nuevo. 

Item 16 
Figura 

Tiempo Limite: 90 segundos 
Ahora trata de hacer este. 

Comience a tomar el tiempo. 

(9) Punto de Decision - Edades 7.0-12.1 1 (despues de administrar el item 16). 

Matrices 

Materiales: Booklet 2 

Comienzo: Todas las edades 

Eiemplos A.B v C 

Booklet 2: Senale la matriz en la parte superior de la pagina y diga: 
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Mira estos patrones 

Senale el espacio en bianco y diga 
Como puedes ver, les falta una parte... 

Haga un gesto circular alrededor de las respuestas en la mitad inferior de la pagina y luego senale el 
espacio en bianco y diga: 

Senala la figura que debe ir aquf arriba. 

Si el nino responde incorrectamente o no responde, de las siguientes instrucciones de enseiianza : 
Eiemplo A 

Este no esta del todo correcto. Hay un ci'rculo en cada parte, de modo que aqui 
tambien debe haber un circulo. 

Eiemplo B 

Este no esta del todo correcto. Arriba hay dos Xs y abajo hay un cuadrado, de modo 
que aqui tambien debe haber un cuadrado. 

Ejemplo C 

Este no esta del todo correcto. En este lado (senale el lado izquierdo) hay cosas iguales. En el 
centro hay puntos, y en este lado (senale el lado derecho) hay lineas. De modo que aqui tambien 
debe haber una linea asi. 

Edades 5.0-711 continue con el Item 1 
Edades 8.0-10.11 continue con el Item 5 

Edades 11.0-17.11 administre el Eiemplo D y continue con el Item 15 
Itemes 1-7 

Ahora mira estos patrones. (senale la matriz) Que le falta aqui? (senale el espacio en bianco) 
Encuentralo aqui abajo. (senale la parte inferior de la pagina) 

Para los siguientes itemes puede decir: Ahora trata de hacer este. O simplemente presente el 
siguiente item. 

Eiemplo D 

Administre como se describe para itemes 1-7. De las siguientes instrucciones de enseiianza a todos 
los ninos. 

(3) Si el nino responde incorrectamente o no responde, senale la matriz y diga: Este no esta del 
todo correcto. La fila de arriba tiene un cuadrado, un triangulo y un circulo. La segunda 
fila tiene las mismas cosas en diferente orden. La Ilia de abajo tiene un triangulo y un 
circulo pero no tiene un cuadrado, de modo que el cuadrado debe de ir aqui. Algunos de 
los patrones, como este, tambien se pueden hacer en el otro sentido: Si miras a este lado 
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(senate el lado izquierdo) bajando, luego al medio, luego a este otro lado (senate el lado 
derecho) resulta igual. 

(4) Si el nino pasa. Senate la matriz y diga: Esta bien. Algunos patrones, como este, se pueden 
hacer de dos maneras. Cada una de las dos filas de arriba tiene un cuadrado, un triangulo 
y un circulo. La fila de abajo tiene un triangulo y un circulo pero no tiene un cuadrado, de 
modo que el cuadrado debe de ir aqui. Pero tambien se puede hacer en el otro sentido: Si 
miras a este lado, bajando (senate el lado izquierdo) luego al medio y luego a este otro lado 
(senate el lado derecho) resulta igual. 

Itemes 8-33 

Ahora mira estos patrones (senate la matriz) Que le falta aquf? (senate el espacio en bianco) 
Encuentralo aqui abajo. (senate la parte inferior de la pagina). Para los siguientes itemes puede 
decir: Ahora trata de hacer este. 

Item 9 

Si el nino falla el item, use las siguientes instrucciones de ensenanza: 

Este no esta del todo correcto. Arriba hay dos triangulos. En la parte de abajo hay un 
cuadrado, de modo que aqui tambien debe haber un cuadrado. Los de este lado (senate el lado 
izquierdo) estan vacios, pero el de este lado (senate el lado derecho) tiene una cruz, de modo que 
aqui tambien tiene que haber una cruz. 

(10) Punto de Decision - Edades 5.0-7. 1 1 - (despues de administrar Item 4) 

(11) Punto de Decision - Edades 8.0-10. 1 1 - (despues de administrar Item 23) 

(12) Punto de Decision - Edades 1 1 .0-17. 1 1 - (despues de administrar Item 33) 

Razonamiento Secuencial v Cuantitativo 

Materiales 

Edades: 6.0-10.1 1 - Sequential & Quantitative Reasoning Booklet (Set A). Lapiz. 

Edades: 7.0-17.11 - Booklet 1 (Set B). 

Comienzo: Edades 5.0-10.11 
Eiemplo A 

De un lapiz al nino. Sehalando el Ejemplo A en el cuademillo diga: 

Mira esta fila de Xs y Os. Forman un patron: Primero un X, luego una O, luego una X, luego 
una O, y asi continua. Que debe de ir aqui? Dibujalo. 

Si el nino falla. ensenele diciendo: 

Aqui va una O porque el patron es X, O, X y O,Xy luego una 0. Tratemos con otro . 
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Eiemplo B 

Pase al Ejemplo B y senalando diga: 

Este patron es diferente. Comienza con O, O, X, X, O Que debe de ir aqui? Dibujalo. 

Si el nino falla . ensenele diciendo: 

En este patron , hay grupos de dos letras iguales: Dos Os y luego dos Xs . Luego deben ir 
dos Os, de ntodo que la respuesta correcta es O. 

Edades 5.0-8.11 Continue con el Item 1 
Edades 9.0-10.11 Continue con el Item 8 
Itemes 1-15 

Todos los itemes 1-15 se administran de la misma manera. En la pagina apropiada, senale el espacio 
en bianco y diga: 

Este es un patrdn diferente. Que debe de ir aqui? 

(13) Punto de Decision - Edades 5.0-6. 1 1 - (despues de administrar item 16) 

Comienzo: Edades 11.0-1 7 . 11 
Eiemplo C 

Senale el par de numeros de arriba y diga: 

Mira los dos numeros en este cuadro. Veamos por que van juntos. El segundo numero es uno 
mas que el primero. (senale el siguiente par de numeros y diga) Ahora mira los dos numeros en 
este cuadro. Van juntos de la misma manera: el segundo numero es uno mas que el primero. 

Si el nino falla . ensenele diciendo: 

Que numero debe ir aqui para que los dos numeros sigan la misma regia? La respuesta debe 
de ser 3 porque en esta pagina la regia es que el segundo numero en el cuadro cs uno mas que 
el primer numero. El primer numero es 2, de modo que el segundo numero debe de ser 2 mis 
1, o sea 3. 

Ejemplo D 

Senale el cuadro de arriba y diga: 

Ahora mira esta p&gina. En este cuadro, el segundo numero es 2 numeros menos que el primer 
numero. (senale el siguiente cuadro y diga) Aqui tambien la regia es que el segundo numero es 2 
numeros menos que el primer numero. 

(senale el espacio en bianco en el tercer cuadro y diga: 

Que numero debe ir aqui de modo que estos dos numeros sigan la misma regia? 

Si el nino falla ensenele diciendo: 
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En esta pagina la regia es que el segundo numero es 2 numeros menos que el primero. El 
primer numero es 4, de modo que el segundo numero es 4 menos 2, o sea 2. 

Ejemplo E 
Diga: 

En cada pagina hay una regia sobre como los numeros en cada cuadro van juntos. Mira el 
cuadro de arriba y el del medio y descubre cual es la regia. Luego dime que numero debe ir en 
el espacio en bianco, de modo que el cuadro de abajo siga la regia. Tratemos con el siguiente. 

Pase al Ejemplo E, senale el cuadro de abajo y pregunte: 

Que numero debe de ir aqui? 

Si el nino fall a ensenele de la siguiente manera: 

Pidale al nino que diga como es que los numeros en los dos primeros cuadros van juntos (o cual es 
la regia). De ejemplos si es necesario. 

Luego pidale al nino que le diga que numero debe de ir en el espacio en bianco. Ayudelo si es 
necesario. 

Edades 11.0-14.11 Continue con el Item 1 6 
Edades 15.0-1 7 . 11 Continue con el Item 24 
hemes 16-39 

Los items 16-39 se administran todos de la misma manera, 

Itemes 16-39 

En la pagina apropiada del Booklet 1 senale el espacio en bianco y diga: 

Que numero debe de ir aqui? 

Si el nino fall en Itemes 16-17 (o 24-25) o en el Item 31, instruyalo ensenandole como en el 
Ejemplo E. En estos Itemes, reconozca las respuestas correctas del nino. 

Punto de Decision - Edades 7.0-10.1 1 (despues de administrar Item 23) 

Punto de Decision - Edades 1 1 .0-14. 1 1 (despues de administrr Item 30) 

Punto de Decision - Edades 15.0-17.1 1 (despues de administrar Item 39) 
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Current education policy advocates family -centered practices over child-centered ones. While 
family-centered philosophies are being adopted in schools, educators are having problems changing 
their actual practices with families. This study reports the results of an applied research case 
study within an ethnically diverse elementary school. Families’ and educators’ perceptions of 
family-centered practices were examined for the purpose of improving practices. Twelve educa- 
tors and 129 families completed a survey measuring perceptions of typical and ideal family- 
centered practices. An exploratory factor analysis was conducted to provide a description of the 
patterning of the families’ perceptions of the school’s family-centered practices. Three main fac- 
tors accounted for 57% of variance, Positive Relating with Families, Partnering with Families, 
and Family-Focused Approaches. There were no differences between educators and families for 
either typical or ideal ratings of family-centered practices. Both educators and families desired a 
higher level of family-centered practices than they were currently providing or receiving. Analy- 
ses considering demographic characteristics indicated significant group differences. The higher 
the education level of families the higher they rated the level of ideal practices desired, but this 
was not so for typical practices. Single-parent families rated typical family-centered practices at 
their school lower than did two-parent families. No differences were found among these diverse 
families by ethnicity or home language. The implications for school psychologists’ work with 
families and schools are discussed. 

Keywords: Applied research, Family, Diversity, Schools, Family-centered practices 

A collaborative report by the U. S. Departments of Education and Health and Human Services 
(Melaville, Blank, & Asayesh, 1993) argues that the child-centered, categorical approach to delivering 
services has a number of shortcomings. These flaws are described as (a) dividing the problems of 
children and families into rigid and distinct categories, (b) being crisis oriented, (c) being unable to 
develop comprehensive solutions, (d) focusing on family weaknesses and problems, and (e) lacking 
functional communication among various agencies. Such problems have led many in the education, 
health, mental health, and social services fields to believe that the key to successfully addressing 
current social problems lies more in how services are provided rather than how much, or how many, 
are provided and that a family-centered approach is necessary (Karasoff, Blonsky, Perry, & Schear, 
1996). 

This approach is based on Bronfenbrenner’s (1979) ecological theory of human development. In 
this theory, he argues that the interconnections between the various systems (school, family, and com- 
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munity), that influence a child’s development, are as decisive for a child’s development as the actual 
interaction the child has within each. He further adds that the capacity of these systems to function 
effectively as a context for the development of the child depends on the existence and nature of the 
interconnections between them. 

Family-Centered Practices 

Family-centered practices have been defined by researchers (Bruder, 2000; Green, Johnson, & 
Rodgers, 1998; Hooper-Briar & Lawson, 1994) as emphasizing families’ strengths rather than deficits, 
promoting family choice and control over desired resources, including the development of a collabora- 
tive relationship between professionals and parents, offering services preventively and to entire fami- 
lies rather than to parents or children separately, and providing a wide array of comprehensive and 
individualized services to families. Me William, Tocci, and Harbin (1998) used qualitative research to 
derive their definition of family-centered practice through exploring the meaning of this type of prac- 
tice from the perspectives of service providers and the families they served. The authors concluded 
that a family-centered interaction style included five main components: family orientation, positive- 
ness, sensitivity, responsiveness, and friendliness. Having a family orientation involved taking an in- 
terest in the family rather than just the child. Positiveness was characterized by thinking the best about 
families without passing judgment on them. Sensitivity was demonstrated by professionals empathiz- 
ing with families and understanding their needs and concerns. Responsiveness to parent requests in- 
volved doing whatever it took to address families’ concerns. Friendliness translated to treating fami- 
lies as friends. Skill in working with children and in collaborating with community agencies to meet 
families’ needs also emerged as an important aspect of family-centered practice. 

McWilliam, Maxwell, and Sloper (1999) specified four assumptions underlying family-centered 
practices; (a) children and families must be viewed as a unit in which an effect on one will affect the 
other, (b) interventions for both family and child will have a larger impact than interventions focused 
only on the child, (c) family members should have a choice about which or what types of services are 
delivered and the amount of involvement they have in those services, and (d) professionals should 
consider family priorities even if they are different from the professionals’ priorities. 

This concept of family-centered practice extends beyond parent involvement and home-school 
partnerships. According to McWilliam et al. (1999), the three concepts are similar in their emphasis 
upon developing positive relationships between professionals and parents, communicating with fami- 
lies, and offering families opportunities to be meaningfully involved in schools. However, they do 
differ in their underlying assumptions and basic values. The concepts of parent involvement and home- 
school partnerships convey a greater value on children than families. Within this framework, school 
personnel view their work with families as a strategy for reaching their ultimate goal of educating 
children. In the family-centered concept, providing support to families is an important goal in and of 
itself. Also, families are seen as the primary decision makers for their children and are supported as 
key decision makers in all aspects of school services, and their needs beyond the education of the child 
are considered. 

In the education field, family-centered service delivery was first considered best practice in early 
childhood intervention (Romer & Umbreit, 1998) and is currently being promoted in elementary schools 
through legislation and policy (Adelman & Taylor, 1998; Johnson, 2000; McWilliam, Ferguson et al., 

1 998). However, many schools and organizations that adopt a family-centered philosophy have diffi- 
culty delivering services in a way consistent with those values (Bruder, 2000; McWilliam, Ferguson et 
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al.). Many service providers have a hard time adjusting to their new role under a family-centered 
approach (McWilliam, Tocci et al., 1998) and report discomfort when parents are put in leadership or 
decision-making roles (Eber, 1996). 

It has been found that school personnel perceive economically disadvantaged and/or ethnic mi- 
nority families as the most challenging to develop relationships with and to get involved with the 
school (Moles, 1993). Therefore, difficulties in implementing family-centered practices may be per- 
ceived by school personnel as being even greater with these particular families. Also, the families’ own 
perceptions of their role with the school could increase the difficulty level of developing relationships. 
Research findings with minority parents suggested that they want to be involved with the school but 
believe it is the school’s responsibility to take the lead in initiating the collaboration (Chavkin & 
Williams, 1993). 

For various reasons an implementation gap often exists between family-centered theory and fam- 
ily-centered practice (Bruder, 2000). In their study of patterns of service delivery and participation in 
family support groups, Green, Johnson, and Rodgers (1998) found little evidence that programs based 
on a family-c entered philosophy actually provide individualized family-centered services consistent 
with that philosophical model. There is also evidence that Individualized Family Service Plans (IFSP) 
still emphasize child-centered instead of family-centered outcomes and family support (McWilliam, 
Ferguson, et al., 1998). 

McWilliam et al. (1999) examined parents’ and teachers’ perspectives of family-centered practice 
in elementary schools (kindergarten through third grade). They surveyed 135 educators and 121 par- 
ents from 88 public elementary schools across North Carolina. Most (approximately 70%) of the par- 
ents had at least a high school education, approximately 73% were Caucasian and the others were 
primarily African-American. Educators and families rated ideal practices as higher than typical prac- 
tices. Families overall rated typical family-centered practices as lower than did educators. These re- 
searchers also found that Kindergarten teachers perceived current services as more family-centered 
than did teachers of third grade classes, but grade level was not associated with families’ perceptions. 
The only demographic predictor of families’ perceptions of typical family-centered practice was so- 
cioeconomic level (SES). Families with higher SES reported current practice as typically more family- 
centered than did lower SES families. 

Purpose of this Study 

Due to the forces influencing educators to change from child-centered practices to family-cen- 
tered practices and the known implementation gap between adopting this philosophy and actual prac- 
tice, it is important that family-centered practices be researched in actual school settings. This project 
was an applied research study in a California Bay Area elementary school with ethnically and linguis- 
tically diverse families. The school participated in the study for the purpose of improving its practices 
with their diverse families. To better understand how to do this, the parents and educators were asked 
to rate current school practices and to report what they think should happen ideally. The school has a 
strong emphasis on serving all students and families equitably, therefore there was great interest in 
exploring how their practices are perceived by various diverse groups of families. The principal was 
specifically interested in variability among the teachers’ perceptions of family-centered practices, as 
well as differences between the educators’ and the families’ perceptions. Therefore the following four 
hypotheses were explored: (a) both educators and families would rate ideal practices higher than typi- 
cal ones, (b) educators would rate typical practices higher than families, (c) educators ratings would 
differ by certain characteristics (grade level taught and years of experience), and (d) families’ ratings 



30 




30 



The California School Psychologist, 2002, Vol. 7 



Table 1 

Measured Dimensions of Family-Centered Practices . 



Item 



Response Statement: Most Family-Centered Practice 



1 . Philosophy 

2. Support 

3. Sensitivity 

4. Responsiveness 

5. Understanding 

6. Empowerment 

7. Communication 

8. Services 

9. Progress Information 

10. Teamwork 

1 1 . Team Meetings 

12. Home Activities 

13. Advocacy 

14. Administrative 



Concern for the needs of all children, including those with disabilities 
and their families is central to the school philosophy statement. 

All school personnel are friendly and supportive of families. 

School personnel almost always respond to families’ needs without being 
critical. 

School personnel almost always make changes when families ask for 

things and almost always check to make sure families are happy. 

School personnel have a good understanding of families’ values and are 
interested in learning even more about families. 

All school personnel make families feel like they do know how to 
help their children learn and grow. 

School personnel are almost always clear when telling families their 
concerns and suggestions and check to make sure families 
understand. 

School personnel give families many choices for their child and family. 

School personnel often inform families about their child’s progress in 
school, good and bad. 

School personnel work together regularly and they almost always ask 
families to work with them. 

School personnel would encourage and help families lead some school 
meetings, if they wanted. 

School personnel give families a choice about whether or not they want 
to plan or do things at home with their child. 

School personnel encourage families to speak up for their child and 
family even with school issues. 

All families are given the option to have at least some participation in 
school administrative activities and decisions. 



would differ by certain characteristics (grade level of eldest child, ethnicity, parent education level, 
family unit structure, and primary language spoken at home). 

METHOD 



Participants 

The sample consisted of 129 families and 12 educators from an ethnically diverse, Title I elemen- 
tary school (K-5) located in the Bay Area of California. The sample of families was composed of 21% 
Hispanic, 19% Caucasian, 17% mixed Asian, 13% Filipino, 13%Biracial, 1 1% African American, and 
6% Other. The majority of the families were two-parent units and the others were single-parent, or 
other type of family, units. Fifty-nine percent of the parents were high school graduates, and 54% 
reported English as the primary language in the home. The other primary languages reported were 
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mainly Spanish, several Asian languages, and Tagalog. 

Of the 12 educators sampled, nine were regular educators, two were specialists, and one was a 
special education teacher. Of the regular educators, five were from the lower grades (K-2) and four 
from the upper grades (3-5). Three of the educators had three years of teaching experience, four had 
four to seven years experience, and five had more than seven years experience. 

Instrument 

The questionnaire used in this study consisted of demographic questions and the shorter version 
of the Family-Centered Elementary School Practice Scale (FCESPS; McWilliam et al., 1999). The 
survey was translated into Spanish and then back to English by different translators to check the accu- 
racy of the original translation. Information about grade level taught and years teaching were obtained 
from educators. For families, information about grade of eldest child in the school, ethnicity, parent 
education level, family unit structure, and primary language spoken at home were obtained. 

The FCESPS is an instrument designed to measure family-centered practices in elementary schools. 
The original 20-item version of the FCESPS contains 6 items specific to special education practices 
and 14 items that address general school practices (see Table 1). Due to the school’s interest in how all 
families view their practices (not just special education parents) the shorter version that only addresses 
general school practices was used. The educator and family versions are worded slightly differently to 
reflect their respective audiences. 

Each item on the FCESPS includes a five-point rating scale (see Table 2). The scales are anchored 
by statements about the level of family-centered practice corresponding to each particular item. The 
first statements represent the least family-centered practices and the fifth statements represent the most 
family-centered practices. There are two scales, typical and ideal, for each item. For the typical scale, 
respondents are instructed to choose one of the five statements that best describes typical practice at 
their school. For the ideal scale, they are instructed to choose the statement that best describes ideal 
family-centered practice. Three examples, of how to do the ratings are given as part of the instructions. 

The FCESPS appears to be a reasonably reliable and valid measure of families’ and educators’ 
perceptions of both typical and ideal family-centered practice in elementary school. Cronbach’s alpha 
for the 14-item version used in this study was .90 for the total typical score and .85 for the total ideal 

Table 2 

Example Item From the Family-Centered School Practices Scale . 



Item = “Responsiveness” of school staff 

Response options (identify what “ typically ” happens at your school and what would he the “ideal " 
for your school) 



1 . School personnel get defensive when I ask for things or tell them my concerns. 

2. School personnel listen but do not do anything when I ask for things or tell them my 

concerns. 

3. School personnel make only small changes when I ask for things or tell them my 

concerns. 

4. School personnel usually make changes when I ask for things or tell them my concerns 

and sometimes check to make sure that I am happy. 

5. School personnel almost always make changes when 1 ask for things almost always 

check to make sure T am happy. 



3 2 
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score, demonstrating internal consistency for both scales. The FCESPS also demonstrated construct 
validity when compared to another measure of family-centered practices (McWilliam et al., 1999). 

Procedures 

The school principal attempted to sample as many educators and families as possible at the school. 
During a staff meeting, copies of the educators’ version of the survey were distributed and then re- 
turned to the office within two weeks. The voluntary and confidential nature of participation in the 
survey project was clearly stated. Several weeks later, the principal visited each classroom to circulate 
the family version of the survey. The survey was explained and then a copy sent home with the oldest 
child in each family. Copies were also mailed to families whose children were absent on that day. 
Families who, according to school records, spoke Spanish as their primary language at home were 
given both the English and the Spanish versions of the survey. The first page of the family version of 
the survey contained a letter from the principal explaining the purpose and nature of the survey. It 
instructed parents to return the surveys to their child’s classroom teacher in an envelope provided 
without names in order to maintain confidentiality. As an incentive, the principal informed students 
that the class that returned the most surveys by the two-week deadline would have an ice cream party. 

Only those surveys that had two or fewer incomplete items on either the typical or ideal scales of 
the FCESPS were used. All of the educators’ surveys and 129 of the approximately 150 surveys re- 
turned by families (approximately a 50% return rate) met the completion criteria for use in this study. 
Incomplete items on the FCESPS, as well as the educational level and family unit categories of the 
demographic section, were dummy coded using rounded mean item responses. Blank items on the 
remainder of the demographics section were coded as no response. 

During coding of educator or family characteristic variables, small groups of similar responses 
were often combined. The educators’ grade taught variable combined into three groups. “Lower Grades” 
contained respondents who taught kindergarten through second grade. “Upper Grades” contained third- 
through fifth-grade educators. The third group was composed of two specialists and one special educa- 
tion teacher. The educators’ years of teaching variable was collapsed into three groups of similar size, 
less than 4 years, 4 to 7 years, and more than 7 years of teaching experience. 

For family characteristics, the home language, family unit, and ethnicity variables contained com- 
bined groups. Under the home language variable, the Spanish-language group contained respondents 
who spoke Spanish and those who spoke both Spanish and English as their primary languages at home. 
Likewise, the Tagalog-language group contained those who were either bilingual in English and Taga- 
log or who spoke Tagalog as their primary language at home. The Asian-language group contained 
respondents who were bilingual in, or who spoke, any Far East Asian language other than Tagalog at 
home. Under the family unit variable, the “other” group contained predominantly singe-parent fami- 
lies but also a few respondents who indicated other nontraditional family unit structures such as grand- 
parents with custody of their grandchildren. Under the ethnicity variable, the Asian group consisted of 
respondents who wrote in or checked off any single Asian ethnicity other than Filipino. The biracial 
group was composed of all respondents who checked two different ethnic options. The other/no re- 
sponse group was composed of the few respondents who indicated other ethnic origins than the main 
groups, or who did not either specify their ethnicity or respond to that item on the survey. 

Data Analysis 

Mean scores for overall typical and ideal family-centered practices were calculated for both fami- 
lies and educators, as well as for various subgroups within these two main categories. For the purposes 
of providing a meaningful way to describe the families’ perceptions of this school’s existing typical 
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family-centered practices, an exploratory factor analysis with orthogonal rotation was conducted. Also, 
in order to explore each of the hypotheses for this study outlined above, the following analyses were 
performed: 

1. To determine whether educators and families differed in their perceptions of family-centered 
practices at their school, two independent two-tailed /-tests were performed, assuming equal variance. 

2. To determine whether typical practice mean scores differed from ideal practice mean scores, 
two-paired, one-tailed /-tests were performed, one for families and one for educators. 

3. To determine if perceptions of typical or ideal family-centered practices among various groups 
of educators differed, a series of four one-way ANOVAs were performed. The characteristics for edu- 
cators examined were grade taught and experience (years teaching). 

4. To determine if perceptions of typical or ideal family-centered practices among various groups 
of families differed, a series of 10 one-way ANOVAs were performed. The characteristics for families 
examined were grade of eldest child at the school, home language, ethnicity, education level, and 
family unit. 

It should be noted that even though the sample of educators is very small, analyses that include the 
educators were conducted in response to the principal’s specific inquiries. Their results are to be inter- 
preted only as exploratory analyses and with extreme caution. 

RESULTS 



Exploratory Factor Analysis 

According to the factor analysis, three factors accounted for 57% of variance. They were labeled 
Positive Relating with Families, Partnering with Families, and Family-Focused Approaches. See Table 
3 for results of the factor analysis, including structure coefficients, communalities, and eigenvalues. 
The Positive Relating with Families factor (Factor I) contained items on personal empowerment, friend- 
liness and support, responsiveness, communication, sensitivity of school personnel, understanding the 
family, and sharing child progress information with family. The Partnering with Families factor (Fac- 
tor II) contained items on teamwork, team meetings, families’ administrative involvement, educa- 
tional activities at home, and family advocacy. The Family-Focused Approaches factor (Factor III) 
contained items on the school’s philosophy of focusing on family needs and concerns and services 
offered for families. 

Families rated current practices in the school highest for Positive Relating with Families (Factor 
I), with a mean score of 3.71, and lowest for Partnering with Families (Factor II), with a mean score of 
3.19. They rated current practices for Family-Focused Approaches (Factor III) in between the other 
two, with a mean score of 3.32. 

Ideal vs. Typical Family-Centered Practices 

Results of /-tests indicated that ideal practice mean scores did differ from typical practice mean 
scores within the groups. The /-scores were as follows: / (128) = 14.8 ,p< .001 for families and / (1 1) 
- 1 3.4,/? < .001 for educators. Therefore, desired (ideal) family-centered practices were rated signifi- 
cantly higher than practices currently experienced (typical practice) for both educators and families at 
this elementary school. The educator and family typical practice mean scores (with standard devia- 
tions in parentheses) were 3.41 (0.38) and 3.57 (0.66) respectively, and their ideal practice mean scores 
were 4.64 (0.28) and 4.46 (0.49) respectively. There were no differences between educator and family 
means for both typical and ideal practices. 
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Educators’ Perceptions by Characteristics 

Results revealed that neither ideal practice means nor typical practice means for educators dif- 
fered by grade taught or by experience (years teaching). Although the typical means were 3.14, 3.54, 
and 3.73 and ideal means were 4.51, 4.68, and 4.79 for lower grades, upper grades, and specialist 
respectively, there were no significant differences among them. For number of years teaching, the 
typical means were 3.44, 3.38, and 3.44, and the ideal means were 4.40, 4.61, and 4.80, for teachers 
with less than 4 years, 4 to 7 years, and more than 7 years experience, respectively. 

Families’ Perceptions by Characteristics 

The results of the one-way ANOVA by family unit for typical practices revealed differences be- 
tween the two-parent family units mean score and other family units mean score. The two-parent 
family units have a higher typical mean score (M = 3.65, SD = 0.64) than other family units (M= 3.37, 
SD = 0.69), F (1, 127) = 5.09,/? = .03. The one-way ANOVA by family unit for ideal practices indi- 
cated that there were no differences between family units. For the two-parent units the ideal mean 
score was 4.46, and for the other family units the ideal mean score was 4.45. 

The parent education level one-way ANOVA for ideal practices revealed differences among the 
various education levels, with the families with higher education levels having higher mean scores for 
reporting their desire for ideal practices. The High School or Less group had the lowest ideal mean 
score (M= 4.33, SD - 0.59), the group with Some College had the middle mean score (M= 4.44, SD 
= 0.47), and the 4-Year College or Beyond group had the highest mean score ( M= 4.61 ,SD = 0.36), F 



Table 3 

Structure Coefficients t Communalities, and Eigenvalues for Factors of the Family-Centered 
Elementary School Practices Scale for Families y Typical Responses. 



Item no. and item 


Positive 

Relating 


Factors 

Partnering 


Family- Communalities 
Focused 


6. Personal Empowerment 


.83 


.17 


.04 


.72 


2. Friendliness & Support 


.80 


.17 


.22 


.72 


4. Responsiveness 


.73 


.19 


.12 


.58 


7. Communication 


.68 


.19 


.28 


.57 


3. Sensitivity of School Personnel 


.62 


.29 


-.40 


.63 


5. Understanding the Family 


.56 


.33 


.21 


.46 


9. Child Progress Information 


.46 


.39 


.13 


.39 


10. Teamwork 


.32 


.73 


-.09 


.63 


1 1 . Team Meetings 


.24 


.69 


-.10 


.54 


14. Admin. Involvement 


.13 


.66 


.41 


.62 


12. Educational Activities at Home 


.08 


.58 


.26 


.41 


13. Family Advocacy 


.37 


.56 


.33 


.56 


1. School Philosophy 


.19 


.10 


.77 


.63 


8. Services Offered 


.42 


.36 


.47 


.53 


Eigenvalues 


5.65 


1.28 


1/05 


— 


% of total variance 


40% 


9% 


8% 


— 
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(2, 126) = 3.65, p = .03. The one-way ANOVA by parent education level for typical practices indicated 
that there were no differences among the education level groups. The means, however, reflected a 
similar pattern with families with higher education levels having higher mean scores for rating typical 
practices; the mean scores were 3.18, 3.40, and 3.53 respectively by level of education from lowest to 
highest. 

The one-way ANOVA results by grade of eldest child at the school, by language spoken at home, 
and family ethnicity revealed that neither ideal practice means nor typical practice means for families 
differed by these factors. For grade of the eldest child, the typical mean scores ranged from 3.90 (K) to 
3.39 (5 th ), and the ideal mean scores ranged from 4. 12 (5 th ) to 4.69 (1 st ). For home language, the typical 
mean scores ranged from 3.57 (English) to 3.68 (Tagalog), and the ideal mean scores ranged from 4.3 1 
(Asian) to 4.51 (English). For ethnicity, the typical mean scores ranged from 3.42 (Asian) to 3.77 
(Biracial), and the ideal mean scores ranged from 4.24 (Asian) and 4.55 (Caucasian). 

DISCUSSION 

This discussion focuses on (a) explanations of the findings, (b) limitations of this study, and (c) 
implications for school psychologists in their work with families and teachers. 

Explanations of Results 

The exploratory factor analysis of families’ typical ratings revealed three main factors in this 
current study, Positive Relating with Families, Partnering with Families, and Family-Focused Ap- 
proaches. Families rated current practices in this school highest for the Positive Relating with Fami- 
lies (Factor I). This suggests that in their current family-centered practices the school staff are per- 
ceived by their families as doing the best in having understanding, sensitive, responsive interactions 
with families; demonstrating positive regard for families’ priorities and values; building on families’ 
strength; and keeping families informed of their children’s progress. The families rated the Partnering 
with Families factor (Factor II) for current practices the lowest. The school staff appear to be per- 
ceived by their families as weaker in forming alliances with families, involving them in leadership and 
decision-making roles, relating to them as equals, and advocating for them when needed. The Family- 
Focused Approaches factor (Factor III) was between the other two and suggests that the school staff is 
currently perceived by families as needing to increase the official, instituted or established policy and 
approaches that emphasize responding to the needs and concerns of their families. It is important to 
note that all three of these factors fell significantly below the level families would ideally like to 
experience at their school. Therefore it is apparent that the families highly value and desire improve- 
ment in all three of these areas. 

Ideal family-centered practices were rated higher than typical family-centered practices by both 
educators and families. These results indicate that both educators and families at this elementary school 
value family-centered practices and perceive current practices as less family-centered than they would 
ideally desire. This is consistent with Me William et al.’s (1999) findings with educators and families 
in North Carolina. 

Also, families and educators at the elementary school in this study had similar values and percep- 
tions of family-centered practices. Educators rated ideal practices at their school slightly higher and 
typical practices slightly lower than families did, but these differences were not statistically signifi- 
cant. McWilliam et al.’s (1999) found that families overall, especially families of special education 
students, perceived typical practices as less family-centered than did educators, but when special edu- 
cation families were excluded, families of regular education students’ scores were not significantly 
lower than educators. This is consistent with the results of the current study. 
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The ethnic groups in the current study’s diverse sample were African-American, Filipino, Asian, 
Hispanic, and Caucasian, and one group of ethnically biracial families emerged; each comprised 11- 
21% of this sample. Also, there were 46% of the families who reported other primary languages than 
English. For this school staff who emphasize equitable services for their students and families, it is 
important to note that no differences based on ethnicity or home language were found in families’ 
perceptions of the school’s typical family-centered practices. These data suggest that no major lan- 
guage group or ethnic group perceives its treatment by school staff as significantly different from any 
other group at their school. 

In addition, there were no differences by parent education level in the ratings of typical family- 
centered practices by the families. However, families’ perceptions of ideal practices at the school 
differed by parent education level, with the higher education levels desiring higher family-centered 
practices. In the McWilliam et al. (1999) study, they found that higher SES ratings were associated 
with higher typical practice ratings. In fact, in their study, SES level was a main predictor of typical 
practice perceptions in families, with SES accounting for 17% of the variance of typical ratings for 
families of typically developing children. (Their study did not analyze this for ideal practices.) The 
authors suggested that the differences in the ratings of typical practices by SES level could indicate 
that families experienced different treatment by school staff based on SES, with higher SES families 
receiving more positive, family-centered type interactions. This does not appear to be so of the school 
in this study and suggests more equitable practices with families across parent education levels. How- 
ever, the current findings do suggest that families with higher education levels do have higher expec- 
tations and desires for family-centered practices at this school. 

Another important finding for the school to be aware of is that families’ perceptions of typical 
practices at the school differed by the type of family unit, with two-parent families rating current 
practices higher than “other” family units, the majority of whom were single parents. Single-parent 
families may have different needs relating to family-centered practices than two-parent families. They 
may have greater needs for family-focused support and services. As implicated in the exploratory 
factor analysis, overall the schools’ families perceive the Family-Focused Approaches area (that in- 
cludes a focus on family needs and services) as needing growth. 

For educators, there were no significant differences in either typical or ideal ratings of family- 
centered practices based on experience or grade level, nor did the families’ ratings imply any differ- 
ences in teachers’ practices by grade level of eldest child. These results support the findings of McWilliam 
et al. (1999) that grade has no effect on ideal practice ratings for educators and no effect on typical or 
ideal practice ratings by families. 

Limitations of the Study 

There are several limitations of this study. As a case study of one school, the general izability of 
results is limited to this school or similar ones. The small sample size of educators, although an ad- 
equate representation of the school’s educators, limits the confidence associated with comparisons 
with other groups. Scaling on the FCESPS was ordinal, not integer. Therefore, ratings might not be 
consistent across items and the distance between whole numbers on any given item might not be equal. 
This would effect interpretation of total mean scores, the scores used for analysis in most of this study. 

Implications for School Psychologists 

School psychologists are very qualified to assume important leadership roles of promoting fam- 
ily-centered practices in schools (Epstein, 1992; Ho, 1997). They understand the importance of the 
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psychological linkages between the support systems of children, acceptance of family differences, and 
identification of, and use of, strengths in families to help children be successful in school. School 
psychologists are also experienced in working with both families and school staff and have the skills 
for program development and evaluation. 

In their role in special education, school psychologists especially need to emphasize the r^l.e of 
families as decision makers in the assessment and IEP planning process and model appropriate family- 
centered practices. It is important, therefore, for school psychologists to personally develop specific 
competencies in providing family-centered services (Ho, 2001; Karasoff, Blonsky, Perry, & Schear, 
1996). These competencies include being able to (a) elicit every family member’s perceptions of rel- 
evant needs, aspirations, strengths, and solutions; (b) move from an exclusive focus on problems and 
needs to a shared goal or vision as to how things could be better; (c) empower families to inventory 
their strengths, preferred solutions, immediate action steps, and barriers to success; and (d) plan for 
services centered around the family. 

Educators in general need to be encouraged to increase their understanding, sensitivity, and re- 
sponsiveness in their interactions with families and to be trained in ways to further demonstrate posi- 
tive regard for families’ priorities and values, and to build on families’ strengths. School psychologists 
are well qualified to provide this critical training for school staff. 

In addition, school psychologists can serve a key role in providing leadership for school personnel 
in conducting applied research for the purpose of developing and improving family-centered practices, 
such as the current study. On a more expansive level they could also lead a school in participatory 
action research to not only develop family-centered practices but family-focused intervention pro- 
grams as well (Ho, 2002). Participatory action research is a process in which researchers operate as full 
collaborators with members of an organization (stakeholders) in linking theory and research to accept- 
able and effective practice. Schools are being encouraged to use this approach to address the chal- 
lenges of developing system-level interventions for many of their current problematic areas, such as 
family-school relationships with economically disadvantaged, and culturally and linguistically diverse 
families. Having the stakeholders (teachers and parents) participate in the development of the family- 
centered practices and intervention programs would result in the acceptability of, commitment to, and 
sustainability of the interventions. 
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There is a growing recognition of the need to provide new teachers with formalized mentoring 
and as a result induction programs are now flourishing around the country. Elements of success- 
ful induction programs include formal mentorship, reflective practice, and collaboration. The 
New Teacher Group (NTG) is one form of induction program that meets these criteria. This 
qualitative study examined how the process of consultee-centered case consultation (CCC) fa- 
cilitated collaboration, problem-solving, and professional development in an NTG. Over the 
course of the school year in this NTG, a developmental shift was evident in which the teachers’ 
descriptions of themselves and their students became more positive and replaced the more nega- 
tive descriptors that were found early in the consultation process. The participants’ negative 
images of themselves, such as inadequate and frustrated, were replaced with positive images that 
conveyed their evolving ability to think flexibly and effectively. The new teachers’ approach to 
problem solving also underwent a change in which they increasingly began to use professional 
concepts and methods to identify and define problems. NTGs offer school psychologists an 
opportunity to expand their consultation role within school districts. 

Keywords: Consultation, Teachers, School Psychologists, Mentoring, Reflective practice, Col- 
laboration 



The first years of teaching are a critical period in the development of new teachers’ ability to meet 
the complex educational and social needs of their students. During this time of entry into the profes- 
sion, novice teachers begin the tremendous task of attempting to meet ever more stringent teaching 
objectives in classes with children from an ever more socially and ethnically diverse student body. 
Beginning teachers often lack the sophistication and professional wisdom necessary to both adequately 
conceptualize the intricate issues inherent in such challenging teaching environments and to imple- 
ment the multifaceted strategies that are required to completely meet the needs of a broad range of 
learners (Hollingsworth, 1989; Kagan, 1992). 

While there is an awareness of the need to actively support the development of novice teachers 
(Cady, Distad, & Germundsen, 1998; Halford, 1998), finding ways to do so has proven difficult be- 
cause of the functional realities of the job: social isolation, lack of opportunities for collaboration, and 
overwhelming time demands. The social isolation and relatively infrequent interactions with colleagues 
discourages genuine dialogue, cooperation, and collaboration among teachers (Rogers & Babinski, 
2002). The lack of opportunities to interact closely with peers limits teachers’ professional develop- 
ment and may undermine their ability to meet the instructional challenges inherent in the work. Around 
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the country, school districts have begun to recognize these problems associated with inadequate new 
teacher induction and have begun implementing various forms of mentoring programs (Corcoran, 
1992; Kestner 1994). 

Key elements of new teacher induction programs include formal mentorship, opportunities for 
situated learning and reflection on practice, and collaboration (Danielson, 1998; Garcia & Harris, 

1 998). Among the most promising kinds of induction programs are new teacher groups (Cady, Distad, 
& Germundsen, 1998; Wiliams & Willamson, 1996). These groups are typically comprised of new and 
experienced teachers and the goal is to use the interpersonal structure to formally facilitate collabora- 
tion, reflection and problem solving. Effective, supportive interpersonal interactions are at the core of 
these groups; therefore, it is crucial that the facilitators be competent group leaders. 

School Psychologists and New Teacher Groups 

Just as the teaching profession has had to adapt its training and expand its functional role to meet 
the educational needs of diverse students, the profession of school psychology has also experienced 
pressures to expand and modify professional practice. Reschly and Ysseldyke (1995) describe a para- 
digm shift in which the professional practice of school psychology has increasingly moved toward the 
provision of services “guided by problem-solving” (p. 17). There has also been an increased focus for 
school psychologists to not only provide direct services, such as assessment, but to also provide forms 
of indirect service, such as consultation (Gutkin & Curtis, 1999). School psychologists have risen to 
the professional challenge and begun to provide various forms of consultation in numerous settings 
(Borgelt& Conoley, 1999; Ross, 1995). 

One new indirect service option available for school psychologists is in the arena of professional 
development through consultation. School psychologists bring a unique skill set to their professional 
workplace and not only are they the most proficiently trained in assessment and child development, 
but they also often have advanced training in group process and system change (Carey & Wilson, 
1995). New Teacher Groups offer school psychologists the opportunity to use their consultation skills 
in the provision of professional development as an indirect service. 

THEORETICAL FRAMEWORK 

New Teacher Groups are similar to other kinds of school-based problem-solving groups in two 
ways: (a) the general underlying assumption is that engaging in collaborative conversations with peers 
facilitates individual and system change (Chalfant, Pysh, & Moultrie, 1979), and (b) the focus of the 
group is on external issues (i.e., children in the classroom). However, they are unique in that a primary 
goal is to support the rapidly evolving professional development of the group’s members. Conse- 
quently, in addition to facilitating collaboration, the process of New Teacher Groups also needs to 
reflect the added focus on the emerging professional. Two primary theoretical constructs underlie our 
conceptualization of the process necessary to assist collaboration and support professional growth: (a) 
Caplan and Caplan’s (1993) framework for consultee-centered case consultation and (b) a socio-cul- 
tural view of situated learning (Bahktin, 1986; Vygotsky, 1978). Taken together, these two approaches 
provided an integrated rationale for the form and process of the unique goal of New Teacher Groups. 
In the next section, each of these theories will be described and their utility for supporting the New 
Teacher Groups explained. 

Consultee-Centered Consultation 

One of the two primary goals of the New Teacher Groups was to provide a formal space in which 



41 




Consultation in New Teacher Groups 



41 



the new professionals could engage in ongoing collaboration and problem solving. Consultee-cen- 
tered consultation was chosen as a facilitative format for the group because of its good fit with this 
goal. The basic tenets of this model of consultation are: (a) that consultation is a problem-solving 
process; (b) it should be conducted in a non-coercive relationship (that is, no supervisory relationship 
between the consultant and the consultees); (c) orderly reflection is encouraged; (d) problems are 
viewed through multiple perspectives and issues are reframed; and (e) alternative hypotheses are gen- 
erated (Caplan & Caplan, 1993; Caplan, Caplan, & Eurchal, 1995). These core characteristics of con- 
sultation overlap with the goal of offering the new teachers a venue for ongoing collaboration. Based 
on the literature on teacher development that the process of becoming a teacher continues beyond the 
induction year (Levin & Ammon, 1992) it was hypothesized that a focus on consultee change would 
be beneficial. 

Socially-Mediated Process 

We also conceptualized the process of consultation and teacher’s development as a sociocultural 
phenomenon (Cole, 1985; Luria, 1976; Rogoff, 1990) because in many ways the task of a teacher’s 
first year is to acculturate into the (sub) culture of professional teaching. A critical developmental task 
for novice teachers is to incorporate and master the knowledge, customs, and habits inherent in the 
society of teaching. The New Teacher Group presented the participants with a safe, culturally im- 
mersed space to question, incorporate, and consolidate their fluency in the culture of teaching. 

In spite of having graduated from a formal training program and acquiring a large body of appro- 
priate content knowledge and skill, teachers in their first years of service often confront novel situa- 
tions, problems, and issues (Rogers & Babinski, 2002). New teachers may be described as still in the 
process of becoming professionals and learning some important aspects of their jobs. So, in addition 
to providing a structured opportunity for collaboration, the New Teacher Groups also provided a situ- 
ated context in which to facilitate the process of change in new teachers’ attributions about and images 
of themselves and their students. The interactive group process combined with the presence of facili- 
tators and peers established a forum that provided for new teachers’ socially mediated development. 

The basic premise of Vygotsky’s theory (1978) of the “development of higher psychological pro- 
cesses” is that higher order learning is fundamentally a socially mediated process. Vygotsky outlines 
three core ideas that describe the theory. First, higher mental functions, such as those used in teaching, 
are social in origin. The process by which individuals acquire advanced cognitive skills is at its core a 
social one, in which intrapersonal psychological development occurs as a result of exposure to cul- 
tural tools (i.e., problem-solving skills) on the interpersonal, social plane (i.e., a New Teacher Group). 
Second, Vygotsky (1978) suggested that signs and symbols mediate higher-order mental functions. 
Knowledge is transmitted from the social plane to individuals via language and other sign systems; 
that is, the judicious and effective use of language will support professional development. Finally, 
supportive facilitators can help people reach higher levels of functioning. 

The basic processes of Vygotsky’s theory capture the interpersonal process within the New Teacher 
Group, and are therefore useful in conceptualizing how the group supported teachers’ learning. From 
a Vygotsky ian perspective, the group can be thought of as a form of situated apprenticeship in which 
the group acts as a mediator of cultural knowledge, where language is used by peers and facilitators to 
guide new teachers. Because New Teacher Groups are a facilitated, social group in which language- 
based interactions are encouraged, the basic conditions for higher learning are in place. In this study, 
the focus is on the process of consultee change as evidenced by the transcriptions of the group meet- 
ings across the school year. 
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THE CURRENT STUDY 

Teacher consultation groups have been used as a means to foster new teachers’ consolidation of 
their nascent “book” knowledge and to facilitate their professional development (Rogers & Babinski, 
2002). The groups have been led by a teacher educator and a school psychology trainer and are based 
on Caplan and Caplan’s (1993) framework for consultee-centered case consultation. The groups pro- 
mote collegial sharing and problem solving by providing participants with opportunities to discuss 
their professional experiences with their peers. The teachers’ perceptions of the groups indicate that 
they feel the groups provide them with social and emotional support and promoted problem solving 
(Babinski & Rogers, 1998). 

In this study, we take a closer look at the new teacher group sessions to determine if the consulta- 
tion groups had an impact on the teachers’ professional development; that is, examining consultee- 
change as the focal purpose of consultation. Previous research (Knotek, 1997) indicates that over the 
course of a school year individuals who participated in consultee-centered case consultation in student 
study teams underwent a developmental progression both in their use of language describing children 
and their concomitant conceptualizations of the children’s functioning. By examining change in the 
teachers’ language over the course of the year we sought to determine if similar developmental pro- 
gressions occurred in the New Teacher Group. Specifically, we focused on the new teachers’ represen- 
tations of themselves and their students, and observed change in their illustrations or what we are 
calling their “images” of children and themselves as teachers. In particular, as we looked at teachers’ 
evolving images, we also focused on their related explanations of students’ problems and their discus- 
sions of potential solutions. 

METHOD 



Participants 

The participants in this study were five white elementary school teachers: four in their first year of 
teaching and one in her second year. Four of the participants were women and one was a man; all were 
in their early to mid-20s. They taught third through fifth grades in four different schools in suburban 
and rural communities. The teachers were invited to participate in the New Teacher Group during their 
orientation programs in their districts. The teachers agreed to meet every other week throughout the 
school year to discuss their issues and concerns with two group facilitators. There were 12 sessions 
over an 8-month period with an average attendance rate of 83%. A problem-solving format was used in 
the group to provide a framework for the discussions. Although the facilitators provided the structure, 
the teachers were free to introduce any topics of concern to them. The topics discussed in the sessions 
and the teachers’ perceptions of the group have been reported elsewhere (Babinski & Rogers, 1998). 

Procedure 

A micro-ethnographic approach, which is an in-depth study of a small number of individuals in a 
naturalistic setting, was used to examine the teachers’ discussions within the consultation group (Bogden 
& Biklen, 1992; Erickson, 1985; Fetterman, 1989). This approach is ideally suited for the study of the 
interpersonal discourse interactions among group members. The unit of study or focal event was de- 
fined as a collaborative discussion in which teachers and facilitators engaged in a problem-solving 
process about issues of concern. 

Five main types of collection procedures were used to gather qualitative data for the study: 
(a) participant-observation, (b) audio recording and transcription of 1 1 meetings, (c) field notes, 
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(d) postgroup reflections, and (e) end-of-the-year interviews with each member of the group that were 
recorded and transcribed. These data sources formed the basis for initial process notes. Bias was 
controlled by cross-checking and “triangulation” (McCraken, 1988) of inferences with other sources, 
including our field notes, postgroup reflections, and the end-of-the-year interviews with each member 
of the group. The data analysis began with an examination of the process notes for evidence of themes 
within and between the meetings. After the themes were identified, coding categories were developed 
to organize the discourse and themes present in the data, and to permit systematic interpretation 
(Boyatzis, 1998). 

Coding Categories 

Coding categories were developed around the content and processes observed in the group’s dis- 
course during the focal event and were both theory and data driven (Boyatzis, 1998). The coding 
categories are divided into three general areas: (a) images of children, (b) images of self, and (c) 
explanations of the problem. 

Within the teachers’ images of children category the discussion ranged from a generalized notion 
of children’s functioning to a more specific description. Generalized descriptors were vague and in- 
definite, including terms such as “bad,” “obnoxious,” and “nasty.” Specific descriptors were words 
that related children’s functioning to some specifiable category, such as, doesn’t pay attention during 
math, and tugs on other children during recess. 

Their images of themselves as teachers included both an action element and a feelings element. 
Within the action element the teachers sometimes considered themselves as incompetent, powerless, 
and perfectionistic, while their images in the feelings element included feeling “miserable,” “trapped,” 
and “overwhelmed” (particularly at the beginning of the school year). We were also able to identify 
two ways in which teachers’ images progressed over time: (a) sense of agency and (b) range of choices. 
The sense of agency refers to the teachers’ description of their experience of interpersonal influence 
between themselves and others (e.g., students or other teachers). For example, if a student was influ- 
encing the teacher, the teacher might say, “He bugs me, he made me really, really angry.” If the teacher’s 
experience was that he or she was influencing the student they might say, “Craig was bugging Mary so 
I had him move to the end of the line where he couldn’t bother her anymore.” 

Range of choices refers to the teachers’ sense of the spectrum of options available to them in any 
given situation. At times, the teachers had a unidimensional view of options that tended to focus on 
problems within themselves or their students, “I have some stubborn children and stressed, agitated 
kids.” At other times, the teachers took a more multidimensional perspective and focused on complex 
developmental or systems issues. For example, “the school is expanding, there’s a shortage of space, 
and consequently people seem to be taking care of themselves first.” 

The final coding category dealt with the teacher’s attributions of the problems. Weiner’s (1986) 
concept of attributions was used to describe whether the teachers were appealing to internal and/or 
external explanations for the issues that they presented in the groups. Included in the teachers’ attribu- 
tions were their sense of their own involvement as well as their students’ involvement in the situation. 
A teacher might describe a problem as some personal, constitutional trait “I just get so frustrated that 
1 lose control of the class,” or as an external issue “The administrators just aren’t helpful.” And, the 
teachers also ascribed the same kind of internal and external locus-of-control to their students, “He’s 
been real depressed and can’t concentrate in class” or “ When Sara comes in I can tell by how dirty her 
clothes are what went on at home last night.” 

These coding categories were used to organize the group’s discourse and to allow for a careful 
analysis of the dynamics contained in the utterances between the group members. 
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RESULTS 



A Developmental Progression 

An analysis of the discourse from the transcribed group sessions indicated that the teachers’ im- 
ages and conceptions of children in their classroom as well as their views of themselves as teachers 
followed a developmental progression over the course of the school year. The group’s initial images of 
children and of themselves as teachers tended to be negative. Early in the school year, the teachers 
tended to lump together all of their students and consider them as unruly groups that freely exerted their 
will upon the teachers. Teachers’ images of themselves were no less problematic. In the initial sessions, 
the teachers evoked images of isolation, inadequacy, and frustration. 

However, as the group collaborated and co-constructed their understandings and interpretation of 
the issues, both within sessions and across the school year, these negative images began to recede. As 
the year progressed, the teachers began to see the children more empathetically and as individuals. The 
group’s collaborative interactions provided multiple perspectives, assisted inquiry, and shifted the role 
of facilitation to the teachers. These interactions served to alter the images depicted in the group. As the 
collaboration moved forward, a developmental progression became evident in which images of chil- 
dren moved from vague generalizations to specific descriptions, and the images of teachers moved 
from a focus on internal self-deprecation to an external focus on developmental and ecological descrip- 
tions of the issues and solutions. By the end of the school year, the images recorded in the transcripts 
are full of a grounded sense of agency and power. The following section provides a case study of 
Annette as an exemplar of the New Teacher Group’s participants evolving images of children and self. 

Initial concerns. During an early group meeting the teachers discussed some of their most imme- 
diate frustrations and problems. Within minutes of the start of the first group session, the teachers were 
using personal speech and presenting each other with graphic images of themselves and of the students 
they served. Because these initial sessions dealt with issues the teachers had found unmanageable their 
initial images of the children and themselves tended to be pessimistic. The students were portrayed in 
an undifferentiated state, as an aggregated assembly. Below is an example from the beginning of the 
first session. 

Annette is a new teacher of a combination 3 rd and 4 th grade class who has just said that much of her 
day was, “. . . children being mean - not good to each other all day.” 

Annette; They just sit there and yell at each other and do whatever they want to do. 

Leslie (a facilitator): It is more of [a problem with] the younger kids or more personality [con- 
flict]? 

Annette: No, it’s all of them. It’s just constant. 

In this exchange the new teacher, Annette, presents an image of the students that is undifferenti- 
ated, and in which the students are combative, inconsistent, and persistently frustrating. The students in 
her class are described as a generalized group (“They just sit there” and “No, it’s all of them”) who are 
a continual problem (“it’s just constant”). Annette’s image of children was also intertwined with her 
image of herself as a teacher - not only did she not differentiate among the children, but also she did not 
initially differentiate herself from the students. The mood and tone of her descriptions of the students 
were mirrored in the mood and tone of her description of herself. Correspondingly, in these next utter- 
ances, which were continued from the previous quote, Annette goes on to present an image of herself 
that reflects a sense of inadequacy and of. falling short of her own expectations. 

Leslie: Has anything helped? 
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Annette: We have great mornings [in my class]. I just can’t figure out what 
makes the difference between a great morning and a horrible afternoon. We have 
a good time, which is usually good, and bad times which are usually bad, but 
can’t figure out what the difference is. I don’t feel like there is a difference in me. 

Annette: [The bickering is] just constant. If it’s not his fault, it’s that one, and not 
that one, it’s this one, etc.. . 

Leslie: Do you feel like you are putting out fires? 

Annette: Yes 

Annette’s description of the issue is phrased in personal speech in which she is powerless to stop 
these difficult children from acting out, . .but I can’t figure out what the difference is. I don’t feel like 
there is a difference in me.” Upon further questioning about the situation in the classroom, Annette 
then describes how this image is tied to her understanding of the situation. 

Leslie: So they’re not just doing this to bug you? 

Annette: No [they’re not just misbehaving to bug me], it is not fair for me to 
ask them to put other things under cover [to make the students just wait for 
convenient time to work out an interpersonal issue] that, I feel I couldn’t put it on 
hold if I were them. I know I could handle it more maturely, but I couldn’t put it 
on hold. 

Here Annette presents an image of an internal conflict in which her power to control her students 
is mediated by her empathy for the children’s feelings. The end result is that Annette has restricted her 
range of available options and feels powerless to intervene effectively. Fortunately, the session does 
not end with Annette holding these images. As the session continues, collaboration unfolds and the 
images begin to change. 

Active collaboration begins . At this point early in the session, Annette has done most of the talk- 
ing and she has painted a picture of herself as a concerned, frustrated teacher who has yet to effectively 
respond to children who are being mean to each other. Next the facilitator begins to ask the other 
teachers for their point of view, “How do you handle it?” And, as other teachers contribute to the 
process, Annette’s images show their first evidence of change. 

Annette: It seems to me like it’s a whole bunch [of students who are acting out in line], maybe it’s 
just an overbearing situation. (Conversation continues. . .) 

Craig: Like when are the certain situations when it actually happens? 

Annette: Like I said, we have 75% of our day, which is really good, and then it’s 
just one part of the day. It could be a horrible morning and a terrific afternoon; it 
could be a terrific morning and a horrible afternoon. 

Craig: It is not consistent, right? 

Annette: No, maybe I’m not consistent in methods , but it’s not consistent. I don’t 
know, I really can’t figure it out. Some days would be perfect and the next day 

and really I am just sorry but praise does not work with this group. [If I say] 

good job [they are] very obnoxious and you are always here to reinforce good 
[children] being good. And, then it’s like we don’t have to be good anymore. 

Craig: I purposely think with things like that it is best to change the environment 
before anything else. Always try to focus on a kid or whatever situation , and try 
to alter that. 

Although Annette continues to use potent generalizations such as “overbearing situation” to por- 
tray the scene in her classroom, her image of herself and the students begins to shift. The other teachers 
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in the group, Craig and Victoria, collaborate with Annette and use professional speech to press her to 
get specific (“When are the certain situations?”) and her speech changes from describing her internal 
states to instead identifying some external skills, “No, maybe I’m not consistent in methods. ” Annette 
has begun to unpack her gloomy images and her explanation of the problem has changed. 

With the narrowing of her focus and an adjustment of the description of the problem to leamable 
skills (methods) comes an opportunity for new images of children and teachers to emerge. The images 
of children and the images of teacher are intertwined and based upon the teacher’s feeling of control 
over the situation. Craig illustrates just such a connection when he says, “change the environment” and 
“focus on a kid ” Craig uses words that present an image of a teacher who has agency, “1 purposely 
think,” and “alter that,” as well one who portrays a value neutral issue “environment,” “situation,” and 
“a [single] kid.” 

Multiple perspectives. As the session continues, the focus of the collaboration shifts toward a co- 
construction of alternative explanations for the problems in Annette’s class. The interpersonal, social 
space of the New Teacher Group is providing a forum for “dialectical constructivism” or interpersonal 
bootstrapping (Marshall, 1992; Vygotsky, 1978). The facilitator begins to pose questions that call upon 
Annette and the group to interpret and explain the process in Annette’s class from multiple perspec- 
tives. Consequently, the images of students and teachers continue to evolve. (The teachers begin to 
more actively assume the role of facilitators in later sessions.) 

Annette: It’s really my younger ones. The “not fair thing” is coming from the 
eight-year-olds. We need rules. 

Leslie: It is so typical that things have to be black and white for these kids and 
that’s the way they think. 

Annette: It has to be a wrong or a right. 

Leslie: Right out of the theory. That’s exactly right. 

Leslie: You know, you are the authority so you do have that right. 

(Intervening conversation) 

Annette: I can do that with my fourth graders. . . . They would be upset but they 
would get it. I don’t know if the third graders would be able to get it. 

(Intervening conversation) 

Annette: No, I think they know, they know [when they are being mean]. 

Especially, my fourth graders know when they have done something wrong and it 
is almost, okay you’re right [and] I’ll deal with the consequences sort of thing. 

But, my third graders, its just that its different. 

Leslie: It is interesting because that is that developmental change you know. 

Seven [-years-old] and eight-years-old, and then older kids. So do it. 

[Understand] that it is not you. It is eight-year-olds. 

Annette: It is a very transitional age, they are very different . . . 

In the above transcript professional discourse, in the form of developmental explanations for 
differences in the students’ functioning, saturate the conversation (“don’t know if the third graders 
would get it,” “developmental change,” and “transitional age”) and Annette incorporates this outlook 
into her speech. The image of children being presented now is of students who have different needs to 
which the teacher must try different approaches. 

In summary, even within the first session there was a pronounced shift in the images of children 
and images of self as teacher that were expressed in the group. The images of children have moved 
from undifferentiated to more differentiated, and the teachers’ image of self has moved away from 



47 




Consultation in New Teacher Groups 



47 



frustrated and powerless to thoughtful and insightful. This initial collaboration has allowed the group 
discourse to move from monolithic and single-dimensional images to slightly more open and multi- 
facetted representations. This example provides evidence that consultation can impact teachers’ under- 
standing and perceptions of school-related issues. Later in the school year, near the end of March, 
several changes are noted in the way the teachers present images of children and images of themselves 
as teachers. 

Annette s new images. The new teacher group was dynamic and it evolved over the school year. As 
the group progressed, so did the teachers’ interwoven images of themselves and their students. Over 
the course of the sessions, the teachers began to change their combative “us versus them” stance 
towards the students. Along with the shift in perspective, there was a noticeable difference in the tone 
of the representations the teachers evoked during the collaboration. 

In the next section, a conversation is presented with Annette about a child in her class who is 
having difficulties with her classmates. As compared to the discussion outlined above, Annette has a 
different perspective of her students and of herself as a teacher than the one she presented at the 
beginning of the school year. Recall that in the first session Annette had a rather downbeat assessment 
of her class and used descriptors such as, “horrible mornings” and “horrible afternoons,” to describe 
the climate of her room. In this series of transcriptions, Annette uses much more positive imagery than 
in the first session as she talks about the difficulties one of her female students is having appropriately 
socializing and playing with other students. 

Annette: I have a child that is similar [who bothers and doesn’t get along well 
with the other students.] What is hard for me is that yes, kids don’t react well to 
her, but it’s because she doesn’t react well to them. It’s not that they discredit her 
from the beginning. She’s given them every opportunity not to enjoy her 
company. It’s just sad because there’s this child in the room and there’s this child 
destined, not destined, but is this child always going to be this way? Are they 
always going to have these problems? Are they always going to be the outcast? 

The sad part is that she is doing it to herself. It’s not that she is [purposely] acting 
to cause these things. It’s not that the kids are mean and ganging up on her, it’s 
not anything like that. It’s that her behavior is not appropriate. 

Victoria: And, until she changes that behavior, the pattern won’t change. 

Annette: She just doesn’t see it. 

Annette begins the description of her student by citing an internal reaction that she is having about 
the student’s situation, “What is hard for me is that kids don’t react well to her.. and “It’s just sad.” 
Instead of negatively evaluating the student as being mean or difficult, Annette tolerates the ambiguity 
of the girl’s situation and displays empathy towards the girl’s predicament. Next, Annette describes the 
complex cycle the girl finds herself in - people aren’t nice to her and she is not nice to other people, 
and wonders about the inevitability of the girl’s future, “...is this child always going to be this way?” 
Because Annette is able to tolerate the ambiguity of this girl’s future she does not need to summarily 
dismiss this child. Annette is able to keep her sense of mastery intact and her freedom of choices open 
as she wrestles with the uncomfortable realities this girl presents. Finally, Annette comes to the conclu- 
sion that the girl’s behavior, not her being, is the issue. 

The image of children Annette presents above is one in which they are complex beings, who have 
their own troublesome realities, who feel pain, and who are worthy of our understanding. The image of 
a teacher that Annette illustrates is of a person, who is flexible in thought, can feel empathy, and who 
can differentiate behavior from essence. • 
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In the next transcription, Annette hypothesizes about how a little girl could have come to such a 
state that she is unaware of basic social cues and lacks the fundamentals of an efficient social reper- 
toire. 

Annette: That’s where you pick up on all those cues [when you are in 
unstructured, child-governed play]. This is where you learn [that] being held 
[roughly] like this is not fun. [Other people don’t] think that this is fun. What is 
fiin [anyway]? You can’t [free play] in ballet or soccer. [Unstructured] play 
[needs to be available] in 3 rd grade kinds of games. . . . even 4 th and 5 th grade. 

How do ten-year-olds play with each other? Not at soccer or ballet or 
gymnastics.... How do you just go on with an eleven-year-old? I don’t think 
there’s enough time for [free play.] 

Annette turns a developmental lens on the reasons for poor social skills. From this perspective, 
Annette is suggesting that children are denied the proper opportunities to learn the basics of play and 
relationships. Instead of blaming children for being “brats,” Annette affiliates with them over the lack 
of appropriate circumstances to learn. Her image of children is that they are overcontrolled to their 
detriment, but are teachable and able to socialize if given the chance. 

Annette: They know and basically they just said Mrs. Paul, “Do you not see 
what she does? This is what she is doing to us.” It is not that she knows [she’s 
hard to play with.] [The other students and I] talked about, how bless her heart, 
on picture day her Mom did her hair. Her hair was sticking straight up and it was 
not [pleasant to look at.] She wore her hood all day long so people wouldn’t see 
her hair. The kids were really nice about it. They kept saying, “It’s not that bad. 

It is something special. It’s something for picture day.” The [other students] 
were really good about it. But, bless her heart, she was just so upset. ... 

Annette: [The students] have done, they’ve done reasonable things. 

Trish: The kids [are reasonable?] 

The above image is striking because it illustrates an open, responsive relationship between Annette 
and her students. There is an open dialogue about a difficult girl, the girl’s special problems, and the 
other children’s humane response to this difficult girl. Annette is confident enough of her own ability 
that she can collude with her students to support the troubled child. If Annette did not trust her own 
ability she could not present the other children with the opportunities to learn to be socially respon- 
sible. Annette’s sense of her competence allows her to consider a range of choices in which she can do 
more than complain about children or doubt herself. 

Within this final image, it is evident that there is a teacher who has allied herself with her students, 
who has a grounded sense of power, and who can tolerate and embrace her complex yet all too human 
children. Annette’s students are presented as humane, reasonable, and teachable children. In summary, 
this exemplar illustrates how a new teacher’s conceptions of children and of herself as teacher can 
develop within the supportive and structured setting of the consultation group. And, Annette’s transi- 
tion was not unique among her peers in the New Teacher Group and represents a developmental trend 
that was seen in the other new teachers. 

The New Teacher Group offered a consultative context in which the new teachers solved prob- 
lems in a constructive and situated setting. While consulting with teachers is not an unusual practice, 
the New Teacher Group’s overt goal of supporting the professional development of the consultee is a 
new application of consultee-centered case consultation. Consultation is usually construed to be an 
indirect means to support students and the focus is typically not so explicitly on the development of the 
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consultee. However, in this study client/student change was not the direct focus and was not assessed; 
instead, the New Teacher Group was primarily aimed at supporting the consultees’ (new teachers) 
further adjustment to their new profession and their continued acquisition of professional skill and 
insight. The New Teacher Group was in essence a situated context in which facilitators and peers were 
engaged in the process of cognitive apprenticeship (Rogoff, 1990). Future studies would help to fur- 
ther validate the efficacy of consultation within NTG’s across different settings and contexts. 

CONCLUSION 

In this paper, transcriptions of the group sessions were used to describe and analyze the interper- 
sonal process of collaborative consultation within the New Teacher Group. It was found that over the 
course of a year, new teachers who participated in the New Teacher Group exhibited pronounced 
changes in their images of their students and of themselves as professional educators. The New Teacher 
Group varied from usual consultative practice (Dougherty, 2000) in that the course of action and goal 
was to further apprentice the new teachers in their ongoing process of becoming educational profes- 
sionals (Bush, 1965; Gregorc & Ward, 1977). 

How can school psychologists become involved with New Teacher Groups? It should be recog- 
nized that^ school psychologists already undertake needs assessments, and engage in numerous forms 
of professional development (i.e., in-services) and administrative leadership (i.e., chairing crisis inter- 
vention teams). Given the critical teacher shortage, districts are often receptive to staff proposals that 
will support teacher retention. A simple, district-level, needs assessment combined with a proposal 
may be all that is necessary to gain administrative support for a New Teacher Group. Also, where 
consultation is already a part of a school psychologist’s professional practice, it is possible to start 
informal, if highly targeted (new teacher) consultation groups. 

New Teacher Groups offer school psychologists an opportunity to expand their systems role within 
school districts. By assuming the role of facilitator and using their training in interpersonal communi- 
cation, group dynamics, child development, and learning school psychologists may contribute a criti- 
cal service to fellow professionals and further their professional practice beyond the traditional role of 
assessment. Through the New Teacher Groups school psychologists can offer developing profession- 
als a supportive, dynamic space in which teachers are free to act as learners and are supported to 
continue their professional journey to meet the needs of growing children in demanding environ- 
ments. Finally, by engaging new teachers in a positive consultation experience during the very forma- 
tion of their careers, school psychologists have the opportunity to broaden their base of peers who 
value and make use of indirect services. 
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Previously published research has not moved beyond studying the general association between 
retention and high school dropout. This longitudinal study seeks to evaluate within-group differ- 
ences, exploring the characteristics of those students who are retained and subsequently drop out 
as compared to those students who are retained and do not drop out. A transactional-ecological 
view of development is presented to assist in situating the findings within a framework of long- 
term outcomes across development. The results of this study suggest that there are early socio- 
emotional and behavioral characteristics that distinguish which retained students are most likely 
to drop out of high school. In addition, maternal level of education and academic achievement in 
the secondary grades were also associated with high school graduation status. These findings 
provide information that extend beyond the association between grade retention and later drop- 
out. In particular, this investigation suggests that it is especially important to attend to the socio- 
emotional and behavioral adjustment of children throughout their schooling to facilitate both 
their immediate and long-term academic success. 

Keywords: Longitudinal study, Grade retention, Dropout, Socio-emotional adjustment, Aggres- 
sion, Social skills, Maternal level of education, Achievement 



With a growing emphasis on standards and accountability, it is crucial that educational profession- 
als attend to the research addressing the outcomes associated with intervention strategies and utilize 
this knowledge to inform school practices (Stoiber & Kratochwill, 2000; Kratochwill, Stoiber, & Gutkin, 
2000; Kratochwill & Stoiber, 2000). Retaining children at grade level is an intervention strategy that 
has been steadily increasing throughout the last three decades (U.S. Department of Commerce, Bureau 
of Census, 1966; 1990). Research published in the last decade has indicated that by 9th grade some 
30% to 50% of students will have been retained at least once in their academic careers (Alexander, 
Entwisle, & Kabbani, 1999; McCoy & Reynolds, 1999; Shepard & Smith, 1989). Overall, it has been 
estimated that approximately 2.5 million students are retained each year (Dawson, 1998; Shepard & 
Smith, 1990). This extra year of schooling is estimated to cost U.S. taxpayers over 14 billion dollars 
annually (Dawson, 1998). 
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Several state and federal politicians have sought to end what is known as “social promotion,” 
where a student is automatically advanced to the next grade with his or her peers (Clinton, 1998, 1999; 
Education Week, 1998). This political trend has been perceived by many involved in education as a 
directive to retain students who do not meet or who fall below state performance standards. However, 
research from the past century fails to demonstrate the effectiveness of grade retention for improving 
either academic achievement or socio-emotional adjustment (Jimerson, 2001a, 2001b). 

The Association Between Retention and Dropout 

There are few studies examining the efficacy of early grade retention that extend through high 
school. Those studies that are longitudinal through high school or beyond consistently demonstrate 
that retained students are more likely to drop out than matched comparison groups of equally low- 
achieving but socially promoted peers (Jimerson, 1999). Moreover, there is a substantial amount of 
literature examining high school dropout that identifies grade retention as a predictor variable (Alexander 
et al., 1999; Ensminger & Slusarick, 1992; Grissom & Shepard, 1989; Rumberger, 1987, 1995). 

A recent systematic review of seventeen studies examining factors associated with dropping out 
of high school prior to graduation suggests that grade retention is one of the most powerful predictors 
of school dropout (Jimerson, Anderson, & Whipple, 2002). Each of the seventeen studies found that 
grade retention was associated with subsequent school withdrawal. Several of these studies include 
statistical analyses controlling for many individual and family level variables commonly associated 
with dropping out (e.g., socio-emotional adjustment, SES, ethnicity, achievement, gender, parental 
level of education, and parental involvement). This research review revealed the consistent finding 
that students retained during elementary school are at an elevated risk for dropping out of high school 
(Jimerson et al., 2002). Research indicates that retained students are between 2 and 11 times more 
likely to drop out during high school than non-retained students (Alexander et al., 1999; Bachman, 
Green, & Wirtanen, 1971; Cairns, Cairns, & Neckerman, 1989; Ensminger & Slusarick, 1992; Fine, 
1989, 1991; Grissom & Shepard, 1989; Lloyd, 1978; McDill, Natriello, & Pallas, 1986; Nason, 1991; 
Pallas, 1986; Roderick, 1994, 1995; Rumberger, 1987, 1995; Shepard & Smith, 1989, 1990; Stroup & 
Robins, 1972; Tuck, 1989). In fact, grade retention has been identified as the single most powerful 
predictor of dropping out (Rumberger, 1995). 

Correlates of High School Dropout 

Over five decades of research has revealed several correlates of high school dropout. Prior re- 
search has identified various demographic status variables, individual characteristics, psychological 
and behavioral measures, and family factors associated with withdrawal from high school (Rumberger, 
1987, 1995). Demographic factors include low SES, neighborhood-level variables, gender, ethnic mi- 
nority status, and low parental education (Cairns et al., 1989; Ensminger, Lamkin, & Jacobson, 1996; 
Fine, 1989; Oakland, 1992; Weis, Farrar, & Petrie, 1989). However, these demographic factors do not 
address the dropout process. Achievement problems and failing grades continue to be strong correlates 
(Ekstrom, Goertz, Pollack, & Rock, 1986; Ensminger & Slusarick, 1992; Gamier, Stein, & Jacobs, 
1997; Lloyd, 1978), but these factors may simply be early indicators of dropping out rather than in- 
volved in causal pathways. 

To complete this picture, other studies have identified social and behavioral influences associated 
with school withdrawal such as behavior problems, poor peer relationships, and family level variables 
(Cairns & Cairns, 1994; Ensminger & Slusarick, 1992; Gamier et al., 1997; Parker & Asher, 1987). All 
of these measures have been shown to predict later high school dropout. A limitation to most of these 
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studies is the reliance on survey and/or interview data and few begin in the early years of a child’s 
academic life. Jimerson, Egeland, Sroufe, and Carlson (2000) conducted a prospective longitudinal 
study of high school dropouts and reported an association between demographic, family, individual, 
and school performance variables and later high school dropout by age 19. A strength of this study is 
that these variables were followed throughout the student’s development, from infancy through high 
school. 

It should be noted that many of the factors that predict dropout are known to be interrelated. For 
instance, socio-emotional problems, behavior problems, and low academic achievement are strongly 
correlated with one another. Thus, disentangling precise causation and connections to later outcomes 
can be difficult at best. It continues to be important to further examine primary characteristics that 
have been associated with later school withdrawal. 

Socio-emotional Adjustment and Academic Achievement 

Socio-emotional adjustment and behaviors at school have consistently been shown to be related to 
academic achievement and may contribute to a negative achievement trajectory over time. Egeland, 
Kalkoske, Gottesman, and Erickson (1990) found that children who were classified as acting out or 
withdrawn in preschool had 1st and 2nd grade achievement scores significantly below those of same 
grade children rated as competent. Behavior problems in the classroom have consistently been found 
to be negatively correlated with verbal ability and reading readiness (Richman, Stevenson, & Graham, 
1982). Furthermore, reading problems and antisocial behaviors often co-occur during the early years 
of schooling (Hinshaw, 1992; Loeber, 1990). 

Rutter, Tizard, and Whitmore’s (1970) study suggested that even with IQ held constant, low 
reading skills were more common in conduct-disordered children than in children who displayed no 
behavioral difficulties. In another examination, Horn and Packard (1985) conducted a meta-analysis 
of factors related to learning problems and found that impulse control and internalizing behavior prob- 
lems measured in kindergarten or 1 st grade were as effective at predicting later academic achievement 
as were intellectual ability and language variables. Ledingham and Schwartzman (1984) found an 
increased risk for grade retention and special education placement amongst primary grade children 
who displayed aggressive behaviors. In a longitudinal achievement study, Jimerson, Egeland, and Teo 
(1999) reported that socio-emotional and behavior problems accounted for negative trends in achieve- 
ment trajectories, even when controlling for previous levels of achievement. Overall, the confluence 
of the available research literature indicates that poor socio-emotional adjustment and conduct-disor- 
dered behaviors are associated with past, present, and future achievement trajectories (Hinshaw, 1992; 
Martin & Hoffman, 1990). 

A Transactional-Ecological Developmental Framework 

It is helpful to consider developmental trajectories utilizing a transactional-ecological model, as 
emerging research suggests that high school dropout is best understood as a developmental process 
(Jimerson et al., 2000). The transactional-ecological model of development places an emphasis on the 
bidirectionality between individuals and their multiple environments or ecological contexts (Nastasi, 
1998) across time (Sameroff & Chandler, 1975). The primary focus of the transactional model is the 
contact between the individual and the environment, whereby multilevel ecological systems (micro, 
mcso, exo, and macro) interact to influence student dropout behavior (Bronfenbrenner, 1979). These 
transactions are altered by one another, each subsequently influencing other interactions in an ongoing 
and continuous fashion (Jimerson et al., 2000). As such, from this perspective, behavior is considered 
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a product of the individual’s past and current circumstances, ecological contexts, and previous devel- 
opmental history (Sameroff, 1992; Sroufe, Egeland, & Carlson, 1999). Accordingly, interventions 
designed to influence dropout behavior can occur at any or all of the multiple systems levels in trans- 
actions to effect change in the individual student’s behavior. 

A wealth of previous retention research has found multiple variables within a child’s developmen- 
tal history that contribute to an increase in the likelihood that she or he will be retained in elementary 
school (e.g., parental level of education, parental involvement with school, poorer peer acceptance, 
more problem behaviors; see Jimerson et al., 1997 for further information). 

Thus, it is important to recognize the interplay between a child’s developmental history and school 
experiences, as well as other ecological contexts (Cairns & Cairns, 1994; Dryfoos, 1990; Evans & 
DiBenedetto, 1990; Jimerson, 1999; Kirsch, Jungeblut, Jenkins, & Kolstad, 1993; Kronick & Hargis, 
1990; Sroufe et al., 1999). It is important to note that this transactional perspective does not suggest 
that grade retention alone inevitably leads to negative academic outcomes. On the contrary, high school 
dropout is likely a result of a multiplicity of factors co-occurring throughout development, all of which 
contribute to an increasingly deleterious trajectory over time. There is a developmental tendency for 
numerous factors to reinforce the continuation of a pathway that has already been embarked upon 
(Jimerson et al., 2000; Sameroff & Fiese, 1989; Sroufe, 1997). 

It is clear that particular school, family, and individual characteristics are associated with an in- 
creased likelihood of grade retention (Jimerson, 1999) and these characteristics will subsequently in- 
fluence a child’s developmental and achievement trajectories. Thus, simply repeating a grade is un- 
likely to address the combination of factors that contribute to low achievement or socio-emotional 
adjustment problems, which prompted the decision to retain the student in the first place. Overall, the 
transactional model of development provides a framework that can aid in the interpretation of achieve- 
ment, socio-emotional, and behavioral outcomes commonly associated with grade retention. It also 
serves to emphasize the necessity of effective prevention and early intervention strategies, which should 
be rooted in a systems perspective that focuses upon multiple factors involved in the dropout process. 

The Current Longitudinal Study 

It has been established that there is a strong connection between high school dropout and grade 
retention (Jimerson et al., 2002). This current longitudinal study moves beyond generalities to exam- 
ine specific behavioral and academic variables of retained students in order to increase our under- 
standing of what places children at risk for later high school dropout. Both retained students and 
dropouts present a variety of profiles; however, certain early characteristics may increase the possibil- 
ity that a retained student will drop out. This longitudinal study is the first to explore characteristics 
associated with those students who are retained and drop out, in contrast to those who are retained and 
continue on to graduate from high school. While many studies have demonstrated the strong associa- 
tion between grade retention and dropout, no studies to date have examined within-group characteris- 
tics of retained students to explore processes that may provide further understanding of this associa- 
tion. This 12-year longitudinal study provides information addressing the following questions: 

1. Do family characteristics differentiate which retained students are more likely to drop out? 
Maternal level of education and value of education will be compared between those retained students 
who drop out and those who persist during 1 1th grade. 

2. Do socio-emotional and behavioral characteristics differentiate which retained students are 
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more likely to drop out? Socio-emotional and behavioral adjustment in kindergarten, 2nd grade, and 
8th grade will be compared between those retained students who drop out and those who persist during 
1 1 th grade. 

3. Do achievement characteristics differentiate which retained students are more likely to drop 
out? Academic achievement in 2nd, 4th, 5th, 7th, 8th, 9th, 10th, and 11th grades will be compared 
between retained students who drop out and those who persist during 11th grade. 

METHODS 



Participants 

The current study explored factors associated with longitudinal academic and behavioral out- 
comes of students who were followed from kindergarten through 1 1th grade. Students who had been 
retained in kindergarten, 1st, or 2nd grade (n = 58), either through a transitional classroom placement 
(n = 18) or by traditional early grade retention (n = 40) were included in this study. To reduce between- 
group confounding effects, 14 students were excluded because they had received special education 
services prior to 1st grade, had been held out of kindergarten prior to enrollment, were of minority 
status, had transferred into the school district during kindergarten, or had a substantial physical limita- 
tion. As reported in previous research, there was no difference between the dropout rates of students 
retained in a transitional classroom or by traditional grade retention (19% and 20%, respectively) 
(Jimerson & Ferguson, 2002). Furthermore, previous analyses demonstrate the appropriateness of com- 
bining the transitional classroom and traditional grade retention groups (Ferguson, Jimerson, & Dalton, 
2001). Attrition due to relocation outside of the school district was 20% through the 8th grade. See 
Ferguson (1991) and Ferguson and Mueller-Streib (1996) for additional information regarding the 
research samples. 

Measures 

Mother’s Level of Education and Value of Education. To secure the mother’s level of education 
data, parent surveys were mailed in the Spring of the student’s 2nd grade year and again in the 8th 
grade; there was an 86% response rate. Mother’s level of education was rated using a six-point rating 
scale (i.e., 1-6): “middle/junior high school,” “some high school,” “high school graduate,” “some 
college,” “college graduate,” or “graduate school.” A “value of education” Likert-like rating scale (i.e., 
1-7) was also secured measuring the value that mothers attached to education (i.e., “How important is 
education for your child’s future?”). 

Socio-emotional and Behavioral Adjustment Teacher ratings provided information regarding 
kindergarten students’ aggression and personal-social functioning using the Kindergarten-Personal- 
Social Functioning scale (K-PSF). The 2nd grade teachers were asked to rate, “Does this child exhibit 
‘aggressive’ physical or verbal behaviors?” on a scale of “No,” “Some,” or “Yes” (scored as 0, 1 , or 2, 
respectively). An additional measure of socio-emotional and behavioral adjustment was a “Teacher 
Rating Scale” (Ferguson et al., 2001), which was comprised of six items (Social Skills, Performance, 
Engagement, Success, Self-Esteem, and Attentiveness) utilizing a 1-9 rating scale (very poor skills to 
extremely high skills). The composite of this rating was called Total Teacher Rating (TTR) and was 
measured in the Spring of 2nd grade. An example of one of the “Success” items is, “experiences 
success in classroom academics.” An example of one of the “Self-Esteem” items is, “expresses self- 
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confidence and self-assuredness.” In 2nd grade, a combination score was compiled utilizing measures 
of aggression, counseling referral (dichotomous), and special education referral (dichotomous). 
Aggression was examined by asking 2nd grade teachers to rate, “Does this child exhibit ‘aggressive’ 
physical or verbal behaviors?” on a scale of “No, Some, Yes” (scored as 0, 1 or 2, respectively). In the 
8th grade, multiple core teachers were asked the same question regarding the student’s aggressive 
behaviors as was recorded in the 2nd grade. The mean from the multiple teachers’ ratings was utilized 
in the current analyses. Teachers’ ratings of kindergarten personal-social functioning levels were used 
as a covariate in the aggression analyses. “Needs additional work” endorsements on the three “per- 
sonal-social functioning” items (“I handle problems and frustrations in acceptable ways,” “I have a 
positive self-image,” and “I cooperate with others”) were tallied from semester and year-end kinder- 
garten report cards to construct a variable with a possible high score of six. This index reflects stu- 
dents’ kindergarten personal-social functioning, with higher scores indicating a deficit in personal- 
social functioning (K-PSF), as rated by teachers. 

Achievement. Academic achievement was assessed at various grade levels with a combination of 
measurements throughout this longitudinal study. The Science Research Associates’ (SRA) Survey of 
Basic Skills Series Test was administered in the 2nd and 5th grades. This is a norm-based test that is 
group administered to the students. The Stanford Achievement Test, 8th Edition (SAT) was adminis- 
tered in the 8th grade. National percentile rankings on the achievement composite scores were utilized 
for this study. Composite grade point averages (GPAs) were calculated using 7th , 8th, 9th, and 10th 
grade fourth quarter marks. The Armed Services Vocational Aptitude Battery (ASVAB) was adminis- 
tered in 1 1 th grade; this study used the Academic Ability Composite from the ASVAB. Scores from the 
2nd grade SRA were used as a covariate in the achievement analyses in order to control for prior 
achievement. 

High School Status. High school status was determined in 1 1th grade by examining the student’s 
enrollment records. Each student was classified as either a dropout or currently enrolled. Dropout 
status was defined as a retained student who was no longer enrolled in high school and who did not 
graduate or complete a high school equivalency exam. Overall, among the current sample, 1 9% of the 
retained students dropped out of high school by 1 1th grade, compared to only 2% of the promoted 
students (Jimerson & Ferguson, 2002). 



RESULTS 

A series of Mests were used to examine mean differences between the retained students who 
dropped out and those who remained enrolled at 11th grade (see Table 1). To control for statistical 
Type I error (i.e., reporting a significant difference, when in fact there is no significant difference) only 
results significant at/? < .01 or p < .001 are discussed as “significant differences.” The findings of this 
12-year longitudinal will be presented as they relate to each of the study questions: 

7. Do family characteristics differentiate which retained students are more likely to drop out? 

Mothers’ educational status was found to differ significantly between the two groups, where the 
retained students who dropped out had mothers who reported lower levels of educational attainment 
(t = 2.51,/? < .01). Mothers’ value of education was found to differ at p< .05. The mothers of retained 
students who later dropped out had previously reported a lower value of education for their children 
(/ = 2.34, p < .05). 
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Table 1 

T-test comparisons of family characteristics, socio-emotional/ behavioral indices , and achievement 
measures between retained students who stayed in high school and retained students who dropped 
out 



Stayed In Dropped Out of 

High School High School 



Variable 


M 


SD 


n 


M 


SD 


n 


P 


Family Characteristics 


Mother’s level of education 


3.64 


.83 


42 


2.50 


.16 


8 


.01 


Mother’s Value of education 


.77 


.43 


22 


.29 


.49 


7 


.05 


Socio-emotional/Behavioral 


Kindergarten K-PSF 


00 

rn 


.96 


39 


2.00 


2.40 


9 


.001 


2 nd grade aggression 


.21 


.36 


47 


.50 


.54 


8 


.001 


2 nd grade combination 


.41 


.65 


47 


1.13 


1.12 


8 


.001 


2 nd grade TTR 


111.48 


25.56 


34 


88.14 


35.17 


7 


.01 


2 nd grade TTR self-esteem 


19.09 


4.26 


34 


12.00 


6.35 


7 


.001 


8 ltl grade aggression 


.85 


.74 


46 


1.44 


.73 


9 


.05 


Achievement 


2 nd grade SRA composite 


71.00 


21.40 


44 


59.00 


24.27 


8 


.20 


5 th grade SRA composite 


65.00 


23.64 


43 


49.00 


12.93 


6 


.20 


8 lh grade SAT composite 


49.00 


24.00 


45 


30.00 


19.42 


5 


.40 


7 th grade GPA 


2.95 


.86 


46 


1.46 


1.09 


9 


.001 


9 th grade GPA 


2.77 


.90 


43 


1.28 


.93 


6 


.001 


1 0 lh grade GPA 


2.81 


.81 


38 


1.02 


.49 


3 


.001 


1 1 th grade ASVAB 


58.13 


22.28 


32 


29.50 


10.61 


2 


.05 



Self-esteem from TTR 

2nd grade combination is aggression, counseling referral, and special education referral 

2. Do socio-emotional and behavioral characteristics differentiate which retained students are 
more likely to drop out? 

Several of the socio-emotional and behavioral measures indicated mean group differences. The 
kindergarten personal-social functioning (K-PSF) was found to be significant, with the high school 
dropouts indicating lower personal-social functioning (F = 10.57, p < .01). All of the measures of the 
children at 2nd grade were statistically significant, such that the retained group who dropped out 
displayed more aggression (F = 9.69 , p < .01), had lower self-esteem ratings on both the teacher report 
{F= 18.04,/? < .001) and the TTR (F= 6.95, p < .01), and had higher ratings on the combination score 
(F= 1 1 .04 , p < .001), which included measures of aggression, counseling referral, and special educa- 
tion referral. Although not statistically significant at p< .01, the retained group who dropped out also 
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continued to display more aggression in later years as measured by teacher report in the 8th grade (F = 
4.72,/? < .05). 

3. Do achievement characteristics differentiate which retained students are more likely to drop 

out? 

When examining the achievement variables, no significant differences were apparent during el- 
ementary school (grades 2, 4, 5). However, during junior high and high school (grades 7, 9, 10) re- 
tained students who remained in school received higher grade point averages than retained students 
who eventually dropped out (p < .001). In addition, although not significant at p < .01, the 1 1th grade 
ASVAB scores of retained students who remained in school were higher than the retained students 
who dropped out (p < .05). 

DISCUSSION 

This longitudinal study is the first to examine within-group differences between retained students 
who stay in high school and retained students who drop out of high school. This is an important 
contribution to the literature, as it moves beyond identifying general risk factors towards examining 
specific within-group risk factors of retained students. Ultimately, this may inform and facilitate the 
design of appropriate prevention and intervention programs that may enhance the socio-emotional 
adjustment and educational success of students who are at risk for early school failure and grade 
retention. Within this study, socio-emotional and behavioral characteristics, academic achievement, 
mother’s educational level, and the mother’s value of education, were examined in relation to high 
school dropout among a population of retained students. 

The results of these within-group analyses are consistent with the general dropout literature, which 
indicates that student level variables including lower self-esteem, problematic behavior, and lower 
academic achievement are associated with an increased risk of dropping out. Supporting the findings 
of previous research, family level variables such as lower maternal educational attainment and lower 
maternal value of education also characterized those retained students who later dropped out of high 
school relative to the retained students who persisted. Within this particular longitudinal study, socio- 
emotional and behavioral variables at each age were consistently associated with dropping out. The 
measures of academic achievement during elementary school did not differentiate future dropouts 
from those who persisted. However, in junior high and continuing through high school, the retained 
students who dropped out demonstrated lower grade point averages. Considering a developmental 
transactional framework, these results highlight the need to attend to indicators of low self-esteem and 
aggressive behaviors early in a child’s life to promote later academic success while preventing delete- 
rious outcomes such as high school dropout. 

Continuing this thread of early identification, prevention, and intervention, it may be useful to 
interpret the findings through a transactional-ecological lens. This study and others (e.g., Jimerson et 
al., 2000) indicate that early measurable factors and behaviors are highly associated with later high 
school dropout. The transactional-ecological developmental model views this early developmental 
history as an important influence on subsequent development including both socio-emotional adjust- 
ment and academic success. Without effective early prevention or intervention programs, the develop- 
mental trajectories of children at risk of poor academic performance will likely lead to subsequent 
academic failure, perhaps even high school dropout. Thus, it is important to consider the confluence of 
factors that begin early in a child’s life. Specifically, low self-esteem and aggression can combine to 
promote a negative academic trajectory that leads to later school dropout status. Furthermore, the 
results of this study and others suggest that grade retention is generally ineffective as an intervention to 
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address these early problems, regardless of when the retention occurs. Often grade retention is imple- 
mented as an early intervention to provide a “year to grow;” however, research has consistently failed 
to demonstrate the effectiveness of grade retention in improving either the self-esteem or aggressive 
behaviors of these students (Hagborg, Masella, Palladino, & Shepardson, 1991; Jimerson, 1 999; Jimerson 
et al., 1997; McCoy & Reynolds, 1999). 

Limitations and Future Directions 

While this study is the first of its kind, and the longitudinal methodology and prospective research 
design are advantageous in providing information through high school, there are limitations that should 
be acknowledged. As with most longitudinal studies of retained students, the sample size is relatively 
small. Also, regarding statistical analyses, the use of Mest analysis to compare groups includes the 
assumption of “homogeneity of variance” (the variance of groups is similar); however, several of the 
variables included in this study have considerable differences in the variance between groups [vari- 
ance may be examined by comparing the standard deviations ( SD ) in Table 1]. In addition, these 
results should be considered preliminary until subsequent studies replicate these findings or provide 
further insights regarding the developmental trajectories of children who experience grade retention in 
elementary school. 

The interactions of the individual student with the multiple influences of the school as an institu- 
tion is an important focus. From a transactional perspective, the school as an organization is a salient 
part of each student’s developmental history. Further research is needed to focus upon how the student’s 
educational experience is affected by the multilayered school culture. “Though the individual attributes 
matter, their impact cannot be understood without reference to how they relate to the understandings 
that different students have of events within the institution” (Tinto, 1986, p. 366). The transactional 
model reminds us to consider how school completion outcomes occur within an ecological context of 
the school as an institution (Ruddock, 1996). 

The transactional-ecological perspective regarding student dropout promotes a focus upon the 
multiple processes that contribute to school completion outcomes for students. This involves a host of 
process variables, which prompts many research questions. For example, what processes are relevant 
to those students who voluntarily drop out of school compared to those who are pushed out by aca- 
demic failure over time? What ecological and unique individual variables or attributes impact the 
dropout process? What personal, social, and organizational influences prevent the dropout process? 
What processes influence those who drop out and subsequently return to school? 

CONCLUSIONS 

Ultimately, the research is unequivocal in identifying that grade retention does not appear to ad- 
dress the needs of these students at risk of academic failure. Findings from this study should not be 
misinterpreted as an indication that retention was an effective intervention strategy for the retained 
students who did not drop out of high school. There is a need for further research comparing the 
retained students who completed high school with matched comparison groups of similarly low achieving 
but socially promoted students. This study highlights the association of early socio-emotional and 
behavioral adjustment and high school dropout among a group of retained students. These findings 
have direct implications for school psychologists and other educational professionals. In particular, 
rather than focusing on the unsupported academic intervention of grade retention, it is time to imple- 
ment prevention and intervention programs that have been empirically demonstrated to meet the needs 
of these students in facilitating both positive academic success and socio-emotional adjustment. 
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Best Practices in Assessing Kindergarten Readiness 
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This artile addresses the numerous challenges that have developed in response to the first na- 
tional education goal that states, “All children in American will start school ready to learn.” The 
definition and measurement of school readiness has been surrounded by controversy for decades. 

There is little consensus on how it should be defined and even greater uncertainty on how it 
should be measured. This paper addresses the issues of assessing readiness by reviewing the 
history, research, and methodological challenges relating to screening instruments for school 
readiness and then links these data to a best practices model of screening techniques. 

Keywords: Preschool, Kindergarten, Assessment, Readiness, Screening Instruments 

In the fall of 1 989, the President and governors of the 50 states met for the first Education Summit 
held in nearly 100 years (Meisels, 1998). This meeting established eight “National Education Goals.” 
First among these goals was the following: “All children in America will start school ready to learn” 
(National Education Goals Panel, 1991). Clearly, this is a lofty and needed goal. However, beyond 
broad agreement about the importance of readying children (Bracken & Walker, 1997; Gredler, 1992; 
Lehr, Ysseldyke, & Thurlow, 1987), the readiness waters are murky. There is little consensus on pre- 
cisely what constitutes this term (Scarpati & Silver, 1999) and even greater confusion on how it should 
be measured. 

This paper addresses the definitional and measurement challenges that plague readiness assess- 
ments by reviewing the history, research, and methodological issues relating to screening instruments 
for school readiness. This research is then linked to a best practices model of screening techniques. 

HISTORY OF PRESCHOOL ASSESSMENT 

The history of early childhood assessment is not lengthy. Kelley and Surbeck (1991), in an exten- 
sive review of this history, identified two periods of high productivity. The first period includes the 
contribution of Arnold Gesell who created a “developmental schedule” that contained approximately 
150 items in four areas: motor development, language development, adaptive behavior, and personal- 
social behavior. His work, which spanned almost 40 years, influenced and is still influencing the 
construction of tests for preschool children (Gredler, 1992). 

The second period, during the 1960s, was a time of high productivity for early childhood assess- 
ment (Kelley & Surbeck, 1991). The funding of the 1964 Child Health and Mental Retardation Act and 
of the Head Start and Follow Through Programs created a demand for preschool tests for diagnosis, 
monitoring, and program evaluation (Zigler, 1 998). Assessments were designed to measure the variety 
of domains included in Head Start instruction (e.g., affective, intellectual, psychomotor, and subject 
achievement; Zigler & Styfco, 1994). 

Over both periods, it is evident that the domains measured by these early assessments were child- 
centered and paid little attention to contextual variables. As discussed throughout this paper, the impli- 
cation of this “within child” model is that the burden is placed on the child to be ready for school while 
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largely ignoring the influences of family, community, school, and culture on the child’s performance. 
As outlined in the following section, it is evident that this child-focused model continues to be perpetu- 
ated in current assessment practices. In addition, the psychometric properties of many of these instru- 
ments are alarmingly inadequate. 

CURRENT RESEARCH ON EARLY CHILDHOOD ASSESSMENT 

INSTRUMENTS 



Heterogeneity 

The research on current early childhood assessments illustrates further the myriad of problems 
surrounding these instruments (Bracken, 1987; Lehr, et al., 1987; Thurlow & Ysseldyke, 1979). Two 
surveys, in New York and Michigan (as reported in Bagnato & Neisworth, 1 994), provide insight into 
the range of instruments used for screening purposes. In the New York study, 177 school districts were 
polled to identify the instruments currently being used in their pre-kindergarten screening procedures. 
It was found that 151 separate tests or assessment instruments were reported in use. In the Michigan 
study, 1 1 1 different tests were in use in the state public schools. These studies suggest that the use of 
locally developed instruments may account for a large number of the measures found in these surveys, 
and that there is little consensus on how kindergarten readiness should be defined or measured (Bagnato 
& Neisworth, 1994). 

Technical inadequacy 

Many researchers report that few individual screening measures have the technical adequacy nec- 
essary for early identification (Lehr, et al., 1987; Mercer, Algozzine, & Trifiletti, 1979; Thurlow & 
Ysseldyke, 1979). In 1979, Thurlow and Ysseldyke evaluated the validity, reliability, and norms of the 
most frequently used tests in federally funded Child Service Demonstration Centers (CSDC) and found 
that only seven of the 28 tests reviewed were technically adequate in all psychometric properties. A 
similar analysis by Lehr, Ysseldyke, and Thurlow (1987) using the Handicapped Children’s Early 
Education Program demonstration projects revealed that among the 19 most commonly used devices, 
only three (Vineland, McCarty, and K-ABC) were technically adequate. The criteria used to define 
“adequate” were compiled from several sources, including the Standards for Educational and 
Psychological Tests, and Assessment in Special and Remedial Education (Lehr et al., 1987). 

Tests of cognitive abilities. In a similar study, Bracken (1987) analyzed the 10 most commonly 
used early childhood instruments (five used for educational placement decisions, and five used to 
assess specific skills and/or abilities). The technical adequacy of these instruments was evaluated 
through various indexes of reliability (median subtest reliability, total test internal consistency, and 
total test stability coefficients), subtest and total test floors, subtest item gradients, and provision of 
validity information. For each of these areas, Bracken (1987) outlined minimal standards of technical 
adequacy. He concluded that many of these tests are severely limited in floor, item gradient, and reli- 
ability, especially below the age of 4 years. 

Examining many of the same psychometric properties as Bracken (1987), Flanagan and Alfonso 
(1995) sought to determine whether certain technical limitations of previous instruments were im- 
proved with the publication of new or recently revised intelligence tests. These authors reviewed the 
following tests: Wechsler Preschool and Primary Scale of Intelligence - Revised (WPPSI-R); Differ- 
ential Ability Scale (DAS); Stanford-Binet Intelligence Scale: Fourth Edition (S-B: IV); Woodcock- 
Johnson Psycho-Educational Battery: T ests of Cognitive Ability (WJ-R: COG), and the Bayley Scales 
of Infant Development - Second Edition (BSID-II). 
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Similar to Bracken (1987), Flanagan and Alfonso (1995) found that most of the tests showed some 
of the same inadequacies at the lower end of the preschool age range. Problems with test floors and 
item gradients, in particular, continued to be evaluated as weaknesses for children below the age of 4 
years. Although test-retest reliabilities reported in the respective test manuals appeared satisfactory, 
the authors, pointed out a number of methodological concerns about the design of these test-retest 
studies. These include small sample sizes as well as the use of samples that were either not representa- 
tive of preschoolers, comprised of too broad an age range, and/or included children beyond preschool 
age. 

Flanagan and Alfonso (1995) found two tests, the BSID-II and the WJ-R:COG, to be technically 
adequate across most criteria below the age of 4 years. Additionally, the technical qualities of these 
instruments appeared to be superior to those summarized by Bracken. These authors conclude that the 
technical qualities of the new and recently revised tests for preschoolers have shown improvement. 

Tests of behavior and social-emotional functioning . The evaluative studies of Bracken (1987) and 
Flanagan and Alfonso (1995) were limited to tests of cognitive ability. Bracken, Keith, and Walker 
(1998) examined the quality of 13 commonly used or newly developed instruments designed to assess 
preschool behavior and social-emotional functioning. Using the same criteria as Bracken (1987), Bracken 
et al. (1998) found that the 1 3 social-emotional, third-party assessment devices had more psychomet- 
ric limitations than preschool cognitive ability measures. When comparing more recently published 
instruments to others with older publication dates, it was found that the newer instruments were gener- 
ally more technically sound. This latter finding parallels the work of Flanagan and Alfonso (1995). 
Therefore, despite the substantial limitations among existing early childhood instruments, there may 
be some optimism for improved quality assessment tools developed in the future. However, it is clear 
that these cognitive, behavioral, and social-emotional assessments focus on the child’s abilities rather 
than viewing their performance in context. Little effort has been made to assess environmental factors 
that prove tremendously influential in a child’s development. In addition, it is evident from the hetero- 
geneity of assessments currently in use that a consistent definition of school readiness remains elusive. 

WHAT PSYCHOMETRIC PROPERTIES ARE MOST IMPORTANT 
IN SCHOOL READINESS ASSESSMENTS? 

The research reviewed above outlines numerous problems surrounding the technical adequacy of 
early screening instruments. When choosing a readiness or screening test, standards for professional 
test development must be considered [see Standards for Educational and Psychological Testing; Ameri- 
can Educational Research Association (AERA), American Psychological Association (APA), & Na- 
tional Council on Measurement in Education (NCME), 1999]. Regarding screening tests, the stan- 
dards state clearly that no such test should be used to make any decision about an individual other than 
referral for further evaluation. However, most screening programs today seem to have adopted the 
model that they must identify those youngsters who, if they do not receive special services, are at risk 
for school failure (Thurlow & Gilman, 1999). Therefore, screening programs have assumed the bur- 
den not just of identifying for further assessment those who may have a disability, but also of predict- 
ing which children already have a problem that will continue as the child matures. Given this conflict, 
the issue of predictive validity becomes paramount in choosing a screening test. This next section will 
review critical methodological issues that school psychologists should address when choosing a screen- 
ing instrument for young children. Particular attention will be paid to the issues of predictive validity, 
sensitivity, and specificity. 
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Predictive Validity 

To establish predictive validity, two things are necessary: the number of children identified as at- 
risk and not at-risk by the test, and some measure of performance indicating those children who per- 
formed adequately in school and those who did not (Gredler, 1992). 

Several studies have looked at the positive predictive value (percentage of children originally 
identified as at-risk who later developed problems) of screening measures. Gredler ( 1 992) analyzed 12 
screening measures and found an average positive predictive value of .55 (i.e., 55% of children consid- 
ered to be at-risk later developed problems). In a similar study, Carran and Scott (1992) found an 
average positive predictive value of .65 for eight screening measures. These results indicate that screening 
measures are better at predicting student success than failure. However, this proves problematic be- 
cause school districts “act mainly on the number of children who are classified as ‘at risk*” (Gredler, 
1997, p. 102). 

Very few studies have analyzed the predictive validity of preschool screening on placement deci- 
sions (Thurlow & Gilman, 1999). In one of the few studies conducted (Kochanek & Hennen, 1988), 
preschool screening data obtained in the spring prior to kindergarten entrance predicted special educa- 
tion need two years later. However, it is unknown whether the same outcome would be realized if the 
screening had been conducted earlier (thus allowing time for service provision). It is clear that much 
less attention has been paid to the efficacy of preschool screening than to the efficacy of early interven- 
tion (Lichtenstein & Ireton, 1991; Thurlow & Gilman, 1999). 

Sensitivity and Specificity 

The desirable outcomes of preschool screening reflect sensitivity (referring to the proportion of 
children who actually performed poorly who also were originally selected by the screening measure) 
and specificity (referring to the proportion of children who performed satisfactorily who were origi- 
nally considered not at-risk by the screening test) (Lichtenstein & Ireton, 1984; Gredler, 1992, 1997). 
Sensitivity is a major concern for school psychologists conducting readiness assessments because it 
reflects the extent to which further testing will be given to only those needing it. 

Stevenson, Parker, Wilkinson, Hegion, and Fish (1976) analyzed the effectiveness of a 
prekindergarten battery of cognitive and psychometric tasks and teacher ratings administered in pre- 
school to predict reading and arithmetic achievement in third grade. Results suggest that the index of 
sensitivity is low. That is, the percentage of poor readers actually found at the end of third grade and 
who were originally identified by the preschool measures was less than 40%. Similarly, between 56% 
and 41% of the children predicted to be low achievers performed satisfactorily. When teacher rating 
scales were used, 85% of the poor readers were not identified (index of sensitivity). Further, only 50% 
of those children who were earlier identified by the teachers as being at risk subsequently became poor 
readers. They concluded that more effective prediction could be made from prekindergarten tasks than 
teacher ratings. After 1 st grade, the more effective predictor was derived from scores on prior tests of 
achievement. 

This lack of success is reinforced by the results of the analysis of 33 early identification predictive 
validity studies by Lichtenstein and Ireton (1 984), a comprehensive study by Pianta and McCoy (1997), 
and in a review of 74 studies by Tramontana, Hooper and Selzer (1988). These authors all conclude 
that screening measures do a better job at telling us who performed well and was placed correctly than 
predicting those children who were members of problem groups. These results support the argument 
that screening measures should not be used to make high-stakes placement decisions such as restrict- 
ing entry to kindergarten. 
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Stevenson et al. (1976) further assert that, “batteries of prekindergarten tasks can be used only 
cautiously in identifying children who need help” (p. 398). These authors conclude that the most 
reasonable use of predictive measures, such as psychometric screening batteries and teacher ratings, is 
to identify children who should be considered for further observation and evaluation rather than using 
the results for assigning children to special groups or classes. 

In summary, predictive validity, sensitivity and specificity are key issues in preschool assessments 
and screenings. Because predictive validity requires longitudinal tracking of children or large retro- 
spective data analyses over time, it has often been ignored. Until these methodological issues are 
overcome, it remains crucial for school psychologists to avoid the use of these measures for labeling 
and placement decisions. What follows is a summary of additional methodological issues that should 
be reviewed when deciding on readiness assessments. 

Reliability, Test Floors, and Item Gradients 

Utilizing the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 1999), 
it is evident that reliability, test floors, and item gradients are additional key methodological issues that 
need to be addressed for educational and psychological testing. These issues will be reviewed and 
addressed as they relate to readiness screening. 

Reliability of a test refers to the degree to which a child’s score is consistent (internal consistency) 
and stable (test-retest reliability) across time (Jacob-Timm & Hartshome, 1998). Of particular interest 
to school psychologists is the issue of stability or the degree to which a preschooler’s cognitive test 
scores are likely to be similar from one measurement to the next. Tests with low reliability produce 
proportionately large portions of subtest and composite variability that are due to measurement error 
rather than true differences in the construct (Bracken, 2000). Adequate internal consistency for subtest 
and total test scores allows the school psychologist to assume that the items that comprise the test are 
highly related and measure a similar domain of behavior. During the assessment process, this permits 
a more concise and clear interpretation of test scores (Flanagan & Alfonso, 1 995). 

Another dimension of technical adequacy involves test floors. The floor of a test is an indication 
of the extent to which an instrument provides meaningful scores at very low levels of individual func- 
tioning. In instances when a poor floor exists, scores may become inappropriately inflated and, conse- 
quently, provide misleading information (Bracken & Walker, 1997). This potential shortcoming is 
particularly salient for preschool and kindergarten children because many assessment cases have the 
goal of determining developmental delay based on a significant discrepancy between the referred 
child’s performance and that of same-age peers. Ceilings are not generally as relevant among early 
childhood tests as are test floors because screening measures are used to predict failure rather than 
degrees of success. It is easier to develop suitable items for assessing the upper limits of young children’s 
abilities than it is to create items that discriminate between the lower limits of ability at this age (Bracken, 
1987). 

Item gradients are an additional technical quality that is crucial in early childhood assessment. An 
item gradient refers to “how rapidly standard scores increase as a function of a child’s success or 
failure on a single test item” (Bracken, 1987, p. 322). Ideally, the incremental change in standard 
scores that results from one raw score unit to another should produce a comparable small standard 
score increase. Unfortunately, early childhood tests are notorious for having steep item gradients, with 
correspondingly large standard score changes associated with minor increases or decreases in raw 
scores (Bracken, 1987). Bracken and Walker (1997) note that an acceptable item gradient requires a 
sufficient number of nonredundant test items placed throughout the test. 
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BEST PRACTICES MODEL 

The national goal of having all children ready for school, the effectiveness of early intervention 
and prevention programs, and legal mandates requiring services to preschool children with disabilities 
have forced school psychologists to examine their assessment practices as they relate to the accuracy 
of identification and the utility of assessment findings for treatment planning and evaluation. How- 
ever, research indicates numerous methodological problems surrounding preschool assessments. This 
paper suggests that such historical problems of child-focused assessments and methodological inad- 
equacies continue to be perpetuated in current practice. In devising a best practices model for early 
screening projects, four suggestions are offered: (a) defining the intended purpose of the assessment, 
(b) selecting ecologically focused instruments with multiple raters and follow-up procedures, (c) de- 
termining the process for conducting the assessment, and (d) thinking carefully about how to analyze, 
interpret, and use the results. 

Define the Intended Purpose of the Assessment 

A recent Child Trends Research Brief, “School Readiness: Helping Communities Get Children 
Ready for School and Schools Ready for Children,” summarizes recommendations from the National 
Education Goals Panel (NEGP) related to assessing school readiness. These recommendations stress 
that assessments should be used only for their intended purposes. For example, assessments designed 
to track achievement at the school district level need to differ from the tests used to identify learning 
problems in a particular child. Screening should be viewed as a first step in assessing the needs of a 
child, not the first step in labeling them for school failure (Pianta & McCoy, 1997; Rafoth, 1997; 
Shepard, 1997; Tramontana, Hooper, & Selzer, 1988). 

Select the Instrument 

Recent scholars are advocating the use of an ecological model to guide the assessment of kinder- 
garten children (Vazquez-Nuttall, Nuttall, & Hampel, 1 999). This model stems from research asserting 
that family background factors have been found to be very useful in predicting school achievement. 
Particularly for lower socioeconomic groups, such factors as parent education for mothers and family 
income for fathers have been found to influence home environment, cognitive development, and school 
readiness (O’Brien, 1996). 

Ecologically focused. Vazquez-Nuttall and Nuttall (1999) propose an ecologically based assess- 
ment approach that has been greatly influenced by the work of Bronfenbrenner (1976; see Figure 1). 
As evident from this figure, the innermost circle, which addresses the individual child, has been the 
primary focus of screening procedures. This represents only one-fifth of the model. It is clear from this 
“ecomap” that current assessment procedures have unfairly weighted these child-focused domains 
while largely ignoring family, agency, and cultural influences on a child’s performance. 

This ecological model offers a comprehensive guide to the assessment and intervention of chil- 
dren. It includes different settings, factors, agencies, and people that need to be considered when 
designing a reliable, valid, and useful evaluation and service plan (Nuttall, Nuttall- Vazquez, & Hampel, 
1 999). This type of model recognizes the instability of the very traits most instruments seek to measure 
as a result of developmental bursts and inconsistencies that defy normative charts. As a result, less 
emphasis is placed on inappropriate and methodologically unsound assessments and more attention is 
paid to what parents, teachers and researchers are telling us about these children. 
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Determine the Process for Conducting the Assessment 

The process of conducting the assessment is as important as the assessment itself. Methodological 
problems that plague these assessments indicate clearly that results may not be reliable. Further, in 
keeping with the emerging ecological definitions of readiness, assessments should include multiple 
sources of information over multiple settings. Using single measures restricts the utility of screening 
programs by failing to recognize the degree to which aspects of school functioning can be interrelated 
and differentiated (Pianta & McCoy, 1997). 

Multiple raters . Meisels (1993) has developed a practical assessment utilizing many of these best 
practices suggestions. Termed “The Work Sampling System,” (Meisels, 1992) this approach hinges on 
teacher observation, uses checklists to increase the reliability of observations; and gathers samples of 




Figure 1. 

The ecomap of child and family functioning. Adapted from Nuttall , Nuttall-Vazquez, and Hampel (1999). 
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children’s work to gauge performance. This system is a performance assessment system that offers an 
alternative to standardized tests with young children. Six checklists cover personal/social develop- 
ment, language and literacy, mathematical thinking, scientific thinking, social studies, art and music, 
and physical development. This system provides information in a contextualized format quite different 
from the setting of large screenings where children typically move from station to station in a large 
auditorium. Therefore, results are more valid and predictive of future performance (Meisels, 1995). 
Parents are also part of the assessment process and contribute developmental information and their 
perceptions of their child’s progress. Meisels has found the information gathered through the Work 
Sampling System can be aggregated and analyzed and shows strong reliability and predictive validity 
(Meisels, 1993). 

Multiple gates. As part of the best practices model, it is essential that children judged as “at-risk” 
as a result of such a screening receive appropriate follow-up evaluation. This is especially true when 
screening has typically involved the administration of only one test that is not part of a more ecologi- 
cally based assessment. Good screening programs should be structured in a way that specific follow- 
up evaluations (which may include more in-depth parent interviews, teacher observations, etc.) are 
mandated at specific points or “gates” in the process (Rafoth, 1997; Walker et al., 1988). As described 
by Walker et al. (1988), multiple gating is a procedure that contains a series of three progressively 
more precise assessments, or “gates,” that (a) provide for the sequential assessment and cross-valida- 
tion of multimethod forms of child assessment, and (b) establish a decision-making structure for the 
aggregation of information produced by different assessment sources. The procedure relies on teacher 
judgment of pupil behavior in the first two assessment stages. In stage three, observational data are 
recorded through direct observation and free play settings by a school professional other than the 
teacher (school psychologist, counselor, resource teacher, etc.). Play-based assessment procedures are 
receiving increasing attention as a critical part of this best practices model (for a review, see Athanasiou, 
2000). These types of ecologically focused, multi-rated and follow-up procedures tackle some of the 
capacity challenges that exist with district wide screenings. 

Think Carefully about how to Analyze, Interpret and Use the Results 

Along with considering ecological factors at numerous points, school readiness measures should 
be used to drive planning, not placement decisions. Due to the bursts and spurts in development, 
results must be considered flexibly along with multiple sources of infonnation about the child. Such 
techniques will help to reduce errors that can result from unstandardized instruments or brief encoun- 
ters with the student. 

In summary, the following checklist can be used by school psychologists to develop a best prac- 
tices approach to school readiness assessment: 

□ Technical adequacy - review the predictive validity, sensitivity, specificity, reliabil- 

ity, test floors, and item gradients before choosing an assessment. 

□ Multiple sources - include information on the child, family attributes, strengths, and 

context. 

□ Multiple raters - gather information from family members, teachers, and other 

significant individuals in the child’s life. 

□ Multiple gates - due to significant variations in child developmental pathways as 

well as ongoing changes in family status, children should be screened on multiple 

occasions over time. 

□ Families as partners - include families not only in the information gathering stage, 

but also as key members of the decision-making team. 
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□ Cultural sensitivity - engender a holistic perspective and attempt to understand 

disability, resiliency, and vulnerability in the context of broader socioeconomic, 
religious, and cultural systems. 

□ Coordination - help families link up with additional community-based services and 

programs. In this sense, readiness assessment should be viewed as the initial step 
in service provision. 

□ Multivariate decision making - no single condition, risk factor, or protective factor 

leads irrevocably to a predictable outcome. Therefore, screening models must 
. allow for these multiple sources of evidence to assume different weights in 
decision making over time. 

CONCLUSION 

In conclusion, what has emerged from this research isn’t a clear picture of model instruments but 
rather a model process that begins with the end in mind and incorporates contextual variables at differ- 
ent points. As kindergarten screening has built up a history, it has become increasingly obvious that 
environmental factors are missing ffom the picture. By defining the intended purpose of the assess- 
ment, selecting ecologically focused instruments with multiple raters and follow-up procedures, deter- 
mining the process for conducting the assessment, and thinking carefully about how to analyze, inter- 
pret, and use the results, screening procedures have the best chance of identifying and appropriately 
serving children in need. 
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Using a Formative Program Portfolio to Enhance 
Graduate School Psychology Programs 

Michael Hass and Jan Osborn 
Chapman University 



This article examines the implementation of a program portfolio in a graduate program in school 
psychology. This effort had two broad goals: (a) to diversify the assessment of student outcomes 
and (b) to enhance students’ ability to reflect on their graduate professional education and how 
their experiences, both intellectual and practical, have shaped their professional evolution. The 
portfolio process described in this article was influenced by the work of Donald Schon on the 
reflective practitioner and the notion that professional knowledge and practice is enhanced by 
making what is tacit, explicit. Connecting students with the broader standards of the profession 
and guiding them in the selection of supporting documentation related to these standards were 
major concerns in developing the assessment model described. Incorporating the standards set by 
the National Association of School Psychologists (NASP), the program portfolio was designed to 
facilitate this process of reflective professional development. Ultimately, the portfolio served to 
strengthen the graduate program while developing students’ higher level thinking skills and criti- 
cal evaluation of their skill development. Student response to the process is incorporated in the 
discussion. 

Key Words: Portfolio, Formative assessment, Graduate school, Reflection, Evaluation 



Faculty in the Chapman University School of Education began a process of reflection and renewal 
in the graduate program in school psychology in 1997. The goals of this process were broad and 
included strengthening admission standards, modernizing the curriculum, expanding fieldwork re- 
quirements, and creating a more comprehensive system of student assessment. A key component of 
this renewal centered on the program’s assessment practices, with faculty determining that program 
assessment should have three characteristics: (a) multiple measures of assessment, (b) a focus on a 
cohesive set of professional competencies, and (c) support the School of Education’s mission to pre- 
pare inquiring and reflective practitioners. As faculty worked to create a more comprehensive system 
of assessment, the concept of a program portfolio began to take shape. This article focuses on three 
issues surrounding the decision to institute a program portfolio in order to expand understanding of 
program and student assessment: (a) the process used to implement such an assessment tool, (b) the 
student reaction to and involvement in the process, and (c) the discovery that a program portfolio can 
be a dynamic document — an assessment tool that is both a learning process and an evaluation product 
benefiting both students and the program. 

The most successful school psychologists do not simply possess more knowledge or use more 
techniques than their colleagues. Knowledge and techniques gained through education or experience 
are necessary but not sufficient for good practice in the complex and demanding environments faced 
by school practitioners. Effective school psychologists use their knowledge in creative ways that re- 
spond effectively to the unique demands of a particular situation. In other words, they improvise. It is 
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critical that those who educate school psychologists understand how the quality of effective improvi- 
sation or what Donald Schon (1987) calls “reflection- in-action” can be encouraged in school psychol- 
ogy educators and students. Reflection begins with puzzlement over the complexities of the situation 
that presents itself. School psychologists are continually presented with complex and ambiguous situ- 
ations. As Schon writes, “Stimulated by surprise, they turn thought back on action and the knowing 
which is implicit in action” (p. 50). This reflection in action is bounded by what Schon calls the 
“action-present” (p. 62). An action-present may take place over a matter of seconds or minutes but may 
also encompass months or even years, depending on the duration and pace of the practice. Reflection 
on practices of shorter duration can be developed through the use of role plays, case discussions, 
simulations, and asking students to explain what they are doing and why. A program portfolio is a way 
of encouraging reflection over a much longer period of practice. This longer-term reflection allows 
students to make explicit the tacit understandings they have developed over the course of a graduate 
program. 

THE STATUS QUO: STATIC ASSESSMENT 

Prior to implementation of the program portfolio, school psychology students completed a written 
comprehensive exam in the final semester of their graduate program. During a half-day period, stu- 
dents wrote responses to six questions self-selected from a list of 10 to 12 questions. For several 
reasons the faculty found this assessment process unsatisfactory. The comprehensive exam seemed to 
tap a skill — regurgitating knowledge about a limited number of questions under pressure of time — of 
little practical value in assessing the competency of students or the effectiveness of a program. Stu- 
dents also completed an oral exit interview, as required by the State of California for all credential 
candidates. The questions used in the oral examination were similar but not the same as those used in 
the written comprehensive exam. In addition, there were no clear standards for performance in the exit 
interview, so it was often difficult to communicate to students how they should prepare. Both forms of 
evaluation were superficial and disconnected from the program goals of educating skilled and reflec- 
tive practitioners. 

THE CHANGE PROCESS 

From the beginning of the renewal process, portfolio assessment was a common theme. Faculty 
began by looking at current models of course portfolios as well as program portfolios used in other 
credential programs in the School of Education. A review of the literature produced many examples of 
education portfolios in both K-12 and pre-service teacher programs; however, there was little discus- 
sion of school psychology portfolios. While portfolios are increasingly a component of education 
programs, debate regarding their purpose and value continues (Barry & Shannon, 1997; Meadows & 
Dyal, 1999; Potthoff, Carroll, Anderson, Attivo, & Kear, 1996; Rakow, 1999). Three broad areas of 
portfolio use have been documented: portfolios used for self-assessment, program assessment, and 
external assessment (Barnett, 1995); however, discussion about what that assessment would look like 
and where the focus of control would be were critical questions for the faculty. While examples from 
teacher education programs proved somewhat useful, the particular needs of school psychology stu- 
dents were not addressed. 

Ultimately, discussions lead to the conclusion that while the use of education portfolios expanded 
in the 1990s (Meadows, Dyal, & Wright, 1998; Morgan, 1999), most assessment remains summative 
in nature, occurring after a program of instniction has been completed (Murphy, 1997). Questions 
about assessment as the culmination of the learning cycle lead to the conclusion that assessment con- 
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trolled solely by the assessors rather than the assessed would be less effective in determining both 
student growth and program strength. In this vein, Murphy discusses the “post hoc” and competitive 
nature of many assessment tools: “Advocates of assessment reform question the value of tests that are 
conducted ‘post hoc, 5 at the end of the learning cycle, and used only to rank students relative to one 
another” (p.82). 

THE NEED FOR PROCESS AND PRODUCT 

The change inquiry revealed three core ideas regarding program portfolios. They must be (a) 
dynamic rather than static, (b) formative rather than summative, and (c) structured in a way that con- 
nects them to student course work and the standards of the wider profession of school psychology. 

Faculty concluded that portfolios could serve to develop the higher level cognitive skills of 
metacognition and metacomprehension. Metacognition refers to the knowledge students have of their 
own thinking process (Brown, 1980). Metacomprehension, a related but subtly different skill, is the 
ability to be aware of and monitor one’s understanding of information (Garbarino, Stott, & the Faculty 
of the Erickson Institute, 1992). Portfolios can best serve this purpose when they are dynamic and 
ongoing rather than simply summative; when they are connected to the present lived experience of 
students (i.e., courses and their work demands) and at the same time help move students beyond these 
day-to-day coursework demands to gain perspective on the wider professional world that awaits them. 

Through discussion and research, a solidified rationale as well as a process for a program portfolio 
emerged: in creating the portfolio, learners would engage in a process of reflection and self-evaluation. 
Through the selection of artifacts and writing reflective essays, learners would gain insight into their 
accomplishments relative to the expectations of the profession and directions for future learning. Dur- 
ing the program, the portfolio process could serve as a way for learners and program faculty to assess 
learning in progress. At the conclusion of the program, the portfolio could help learners and program 
faculty assess if program goals and objectives had been accomplished. Lastly, portfolios could be 
shared with persons external to a particular program (e.g., employers, accrediting bodies) to facilitate 
their understanding of learners’ skills and knowledge. 

STUDENT REFLECTIONS 

To include students in the process, they were interviewed early in the stages of creating a program 
portfolio as part of a comprehensive system of student assessment. Student input into the entire pro- 
cess proved invaluable and lead faculty to a number of decisions regarding the program portfolio. One 
area of particular importance involved student feedback regarding a summative portfolio model. Stu- 
dents captured the limitations of a summative portfolio: “If we’re just compiling all those papers for 
whoever is going to ‘assess’ them, that assessment is about organization, about presentation more than 
anything else.” They expressed other concerns regarding the summative model, “If the portfolio is just 
a collection of examples of what I learned to present to an employer, I’m just saving everything; it’s 
paperwork management.” Their questions about employment use provided a theme of discussion, 
“Bottom line, who’s going to look at any of this? No one has ever asked me for a portfolio” and “They 
don’t have time to look at some three-inch binder in a half-hour interview; they want to listen, to see if 
we can express ourselves orally.” 

The student questions prompted further reflection, “What, exactly, would be included in a pro- 
gram portfolio?” Barnett (1995) makes a distinction between two types of evidence that might be 
included in a portfolio — artifacts and attestations. Artifacts are “the tangible products created as people 
complete different tasks” (p. 200). Artifacts typically show rather than tell about a learner’s compe- 
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tence in a domain or area of skill and knowledge. They might include products created as course 
assignments or those completed as a natural part of a learner’s work or field experiences. Coupled with 
reflections about what the articfacts evidence, they can be powerful support documents in a portfolio. 
Attestations, on the other hand, are documents prepared by someone else that validate or witness 
someone’s knowledge or skills. These could include letters of recommendation or formal evaluations 
by fieldwork supervisors. As the program portfolio took shape, primary importance was given to arti- 
facts coupled with reflections and attestations, a less direct type of evidence, serving a useful although 
secondary role in terms of the portfolio. 

FORMATIVE VS. SUMMATIVE PROGRESS EVALUATION 

Rather than serving as a culminating product, however, the program portfolio provides important 
information about student learning throughout their education. Ideally, students begin thinking about 
their portfolios the first day of the program, providing them a structure to document their progress and 
evidence program standards as well as reflect upon their needs and achievements within the program. 
Student involvement in the assessment, each one determining what best provides evidence of meeting 
a standard, adds a personal meaning-making component to the assessment component of any program. 
Students expressed a desire for the formative model even before they were presented with a specific 
structure. In interviews, students articulated their needs: “The portfolio needs to serve us as opposed to 
being this presentation for someone else.” “It could be a resource for me, a tool for us to use.” “Yes, a 
process of acknowledgement of learning, something concrete you take with you to build upon.” 

A formative portfolio can be a meaningful assessment tool, especially when it is held to a high 
standard on its own and designed to enhance and document process as well as serve as a final product. 
This formative as opposed to summative view of assessment is an important distinction. While 
summative assessment is designed as an end-point, formative assessment is designed as part of the 
process, providing an avenue for further reflection and growth (Scriven, 1991). If student learning is at 
the center of a program, then formative evaluation via portfolio provides students a process with which 
to organize knowledge, to reflect upon program standards, and to assess their own progress in light of 
program expectations. However, here again, the tension exists. A paradigm that establishes portfolios 
as a compilation of “best work,” may miss the opportunity for students to reflect upon their practice 
and encourage them to select the products upon which they received the highest grades. A program 
portfolio must provide students an opportunity for reflection, an opportunity to discuss areas of chal- 
lenge and growth as well as success. It must continually go back to the concept of reflection in action, 
providing an avenue for reflection of work, leading to deeper understanding and action. 

Portfolios can provide a dynamic view of assessment, reflecting the idea that learning is more 
richly and accurately portrayed by multiple evidence collected over time (Wagner, Brock, & Agnew, 
1994). But even within the spectrum of those who use portfolio assessment, there is a wide range of 
use; for example, many assessment models fail to take assessment beyond end-of-the-cycle or pre- 
scriptive parameters. In such models, students are given lists of products to include in a portfolio they 
simply put together at the end of the program. Several models of portfolio assessment keep the locus 
of control or expectation in the hands of the assessor. In so doing, the portfolio becomes little more 
than a filing system. 

Schools of education throughout the country are working to use portfolio assessment in more 
meaningful ways. Using Portfolios to Assess the Performance of School Psychology Graduate Stu- 
dents (Prus, Maxton, Thomas, & Robinson-Zanartu, 1996), a 1995 survey of 214 school psychology 
programs, identifies both advantages and disadvantages of program portfolios. Identified advantages 
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included that portfolios facilitated viewing learning over time, helped assess multiple components of 
the curriculum simultaneously, were seen by students as relevant, and opened dialogue between fac- 
ulty and students. Some of the disadvantages cited were that portfolios were costly in terms of time and 
effort, should be cross-validated with other measures, needed to have clear guidelines, and could be 
challenging for faculty. Both advantages and disadvantages were considered in the portfolio change 
process. 

In an attempt to create a portfolio assessment component that accurately reflects the program 
standards as well as provides students with a format for their own collection of work samples, reflec- 
tions upon that work, and ultimate determination of what best reflects their abilities, the Chapman 
University School Psychology Program (CUSPP) began exploring portfolio as both a learning process 
and assessment tool. 

Connection to Standards 

The selection of portfolio domains determines the selection of artifacts and communicates the 
expectations of the program to the student. Although some approaches to portfolio evaluation dictate 
the selection of specific artifacts, this is overly prescriptive and encourages uniformity rather than 
creativity. The portfolio should encourage the creation of a personal narrative about identity, knowl- 
edge, and practice. Such narratives help students reflect upon who they are as professionals, what they 
know, and how they can apply knowledge to practice. 

The CUSPP structured the domains of the program portfolio around broad standards set by the 
National Association of School Psychologists (NASP) in 1994. These broad standards provide stu- 
dents with clarity about program expectations and connects the program to the concerns and interests 
of the broader profession. One student points out this relationship between standards and the larger 
professional arena connection in the portfolio introduction: “I have subdivided my work as each piece 
reflects my learning and experiences in one of the eight domains identified by the National Association 
of School Psychologists.” 

Linkage to Program Course Work 

Each portfolio domain is linked to one or more courses in the program. The courses provide 
knowledge and experiences that learners use in formulating their responses to the Portfolio/Exit Inter- 
view questions. Course products (papers, projects, etc.) become potential artifacts used to support 
learners’ responses to the Portfolio/Exit Interview questions. A paragraph is included in each course 
syllabus explaining the relationship of that course to the Program Portfolio. The following example is 
taken from the syllabus in a course titled, “Introduction to the Ethical Practice of School Psychology: 
Students in “Introduction to the Ethical Practice of School Psychology” should choose 
one or more of their assignments as artifacts for their program portfolios. Instructors 
should structure at least one substantive assignment with this in mind. The focus of 
“Introduction to the Ethical Practice of School Psychology” should be on the portfolio 
domain of Professional School Psychology and could include projects related to such 
topics as (a) history and foundations of school psychology, (b) legal and ethical issues, 

(c) professional issues and standards, (d) alternative models for the delivery of school 
psychological services, (e) emergent technologies, and (f) roles and functions of the 
school psychologist. Artifacts can take many forms but should assist students to sup- 
port the assertion that they are competent in that domain or area. 
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PORTFOLIO QUESTIONS 

The State of California requires that all candidates for a teaching or specialist credential pass an 
Exit Interview. Rather than have two disconnected assessment procedures, the program combined the 
Program Portfolio and the Exit Interview into one evaluative process. Each of the portfolio domains 
has a corresponding series of questions: 

Professional Portfolio and Exit Interview Guidelines Corresponding to Portfolio Domain — 
Psychological Foundations 

The domain of Psychological Foundations includes such topics as biological bases of behavior 
(e.g., biological bases of development, neuropsychology, physiological psychology, and 
psychopharamacology), human learning, social and cultural bases of behavior (e.g., cross-cultural 
studies, social development, social and cultural diversity, and social psychology), child and adolescent 
development, and individual differences (e.g., human exceptionalities, developmental and psychopa- 
thology). 

Guiding Questions 

1 . You have been asked to consult with a sixth-grade teacher who is concerned about a student in 
her class who has problems paying attention during instruction. What theories or principles of learning 
would guide your questions or suggestions? 

2. What academic programs or system level interventions have proven most helpful with students 
who come from economically disadvantaged households? What factors or program components do 
these programs have in common? 

3. Other than income and English language proficiency, what home or family factors seem to have 
the most influence on student learning? 

4. Juan is a first-grade student who moved to the United States from Mexico a month ago. He 
speaks almost no English. Describe the course of his language development over the next few years. 
How long might it take him to develop CALPS? How long might it take him to develop BICS? How 
long might it take him to develop academic proficiency in English? What factors might influence the 
rate and trajectory of his acquisition of English as a second language? 

5. What are the implications of sociocultural, demographic, and lifestyle diversity to the profes- 
sion of School Psychology? 

6. What developmental issues are important to keep in mind when working with a 7-year-old 
second-grade student? What developmental issues would be important when working with a 16-year- 
old high school student? 

7. What are some empirically validated treatments for a common social and emotional problem 
such as depression, ADHD, and eating disorders? 

8. Describe the federal definition of a learning disability. What are some limitations and critiques 
of this definition? 

9. What signs or symptoms would you look for to make a diagnosis of mental retardation, autism, 
and depression? 

These questions, included in course syllabi, clarify the specifics of the domain, assisting the stu- 
dent in the selection of artifacts, and form the basis for the Exit Interview. Artifacts may include class 
assignments or products gathered during fieldwork experiences. For instance, in response to the ques- 
tion, “Describe the federal definition of a learning disability?” students might include a 
psychoeducational report that describes their diagnosis of a student with a learning disability. The 
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portfolio then supports learners’ responses to the questions. Used in this way, the portfolio is a process 
rather than a product. It becomes a dynamic support for learner inquiry rather than something concrete 
and fixed. 

Culmination of the Process 

Organized around the standards of the program, the portfolio process culminates in a product 
expressive of a student’s educational experience and understanding. The candidate must be able to use 
the language of the profession to reflect upon the artifacts as evidence of each domain. It is structured 
to allow student choice and voice. The student uses the portfolio to guide and inspire responses in the 
Exit Interview. It is both a reflection of where the student has been and a tool to help the candidate go 
forward. It is not an artificial product, a notebook stuffed with pre-determined class assignments; 
rather, it is personal and dynamic, used to shape responses in an exit interview as well as a marker of 
progress. 

Students who completed the program with the program portfolio as outlined above were able to 
discuss their understanding of the process, expressing the dichotomy between initial expectations (those 
of a summative nature) — “in the end you will be required to compile a portfolio,” and actual experi- 
ence (deeply formative) — “There was a theme to my portfolio: diversity, being a change agent, going 
beyond the role of a school psychologist. I want that to be reflected in my portfolio.” Of course, this 
understanding did not happen overnight. Students struggled with this type of formative assessment, 
this opportunity to go through a process to gain understanding. Their feedback has been invaluable. A 
participant said, 

There was uneasiness among the students due to the lack of structure. Students 
were not used to the idea of having so much say in their own learning. We’ve 
always been taught; we will complete what is required. The lack of structure was 
not something we were used to. A lot of questioning. Often we would ask, “What 
is it you are looking for?” We wanted direction. You left it open. Ultimately, it is 
our decision what to include, what will reflect our own experiences. 

Prior to using portfolios, student conceptualizations of a final product were well established. They 
asked for the “list” of requirements and were prepared to compile this list and be done with it. The idea 
of actually seeing the portfolio as a process for learning was another matter: “I thought this was going 
to be a weekend thing where I was going to put all the pieces where they fit; kind of like a puzzle. But 
it doesn’t suffice to do that. You find out what you need to tie it all together. Not an end product but 
something cohesive.” The students’ understanding of a portfolio evolved as they participated in the 
process. 

When asked about the transition from initial concept to a deeper appreciation of the program 
portfolio, one student said, 

It was not a matter of grabbing what I needed to reflect this competency or that. It 
was harder than I thought because I had to evaluate the project, evaluating how my 
work most accurately reflected a competency. There was a linkage between a vari- 
ety of projects and the competency. It started to come together. In hindsight, those 
projects were much more representative of a certain competency than just a re- 
quirement to complete the course and receive a grade. It was a continual process of 
coming back, a coming back to look at it. 

The formative nature of this kind of portfolio assessment allowed the students to reflect, to orga- 
nize their knowledge and assess their own progress. It was the student choice that most drove the 
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reflection process: 

A much more complex task than placing the pieces where they belong. What went into 
the portfolio was always my choice as to what reflected a competency. It was always 
my decision. It’s an individual's portfolio. No two portfolios are going to be similar. A 
true reflection of my own experience. 

BEYOND THE CLASSROOM 

This reflection led beyond the boundaries of the graduate school classroom, as it became more the 
students’ self-narrative rather than a narrative about the content of course work. In a reflection, one 
student writes in the ‘Psychological Foundations’ section of the portfolio, “I have included a paper on 
working with Hispanic American families. This paper was written to present at a district in-service on 
cultural diversity and understanding the needs of minority families.” One of the most exciting out- 
comes of the portfolio as both a process and product experience was the realization that students 
discovered themes in their own work, themes that connected learning from course to course. One 
student said, “As I was putting together my portfolio, it became apparent to me that the projects were 
not just the end requirement of a course but were a reflection of what I was interested in and repre- 
sented an overall theme.” 

It is in the discovery of themes that students have gone to another level in their thinking of them- 
selves as professionals: 

It was looking at my work, describing what each piece was and describing how it was 
a reflection of my experience in the program. It was looking at how I could continue 
to learn in the areas I was interested in. It has meaning, it has value to me, and I want 
to continue reflecting upon what I have reflected in my portfolio. 

Having graduated the first group of students with an opportunity to experience the program’s 
portfolio, the duality of the work has been a discovery, for indeed the program portfolio is both a 
learning process and an evaluation product, not an artificial product filled with pre -determined class 
assignments, but a dynamic document that helps a candidate make meaning of the program and the 
profession. 

CONCLUSION 

The process of creating and beginning to implement a program portfolio has itself become an 
action-present. Based on initial experience with program portfolio and reflection on the process, sev- 
eral changes are under consideration. Initially, students included too many artifacts in their program 
portfolio, thus limiting some of the reflection involved in being selective of what products to include. 
To address this, students were advised to limit their selection to no more than three artifacts for each 
domain. The pressure of increased selectivity enhanced the reflective nature of the process. Faculty 
also discovered that after three years of intense involvement, understandings shared between students 
and faculty had become overly tacit, also limiting the reflective quality of the exit interviews. To 
remedy this, the program began to include community members and faculty from other disciplines in 
the interviews. Having to discuss artifacts and respond to questions from “outsiders,” created an im- 
mediate action-present, further enhancing the reflective nature of the portfolio process. Students tended 
to make the portfolio a default-summative task by waiting until the final moments to select and orga- 
nize artifacts. In response, the faculty is creating greater awareness in the program, by including more 
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explicit discussions about the portfolio process early in the program, by referencing the portfolio in 
course syllabi, and by focusing course assignments on the domains. 

The experience of the CUSPP’s work with portfolio assessment also has implications for others. 
Reflection on assessment of both students and program is a necessary starting point. Considering the 
multi-faceted nature of portfolio assessment, determining how a program portfolio might be both for- 
mative and summative is a next step. Such discussions lend clarity to the idea of an assessment tool 
that goes beyond a collection of best works or attestations written by a third party. Talking with faculty 
and students about Prus’s (1996) findings on the advantages and disadvantages of program portfolios 
as well as the portfolio implementation process in this article will help programs make decisions based 
on prior knowledge. One ongoing issue is how to communicate this understanding of a program port- 
folio to students at the outset of the program. Given its reflective quality, this poses a problem with 
new students who want high levels of structure. While introducing the concept of the program portfo- 
lio as students enter the program is desirable, in some ways they are not yet ready grasp the concept. 
Once the idea of a reflective practitioner is established in various courses, they have a richer under- 
standing of portfolio as both product and process. 

As the student voices in this article clearly attest, a dynamic, formative program portfolio is, 
indeed, a tool that can help students make meaning and gain greater insight into the themes they will 
develop in their professional experience. But as we work with the concept of a dynamic assessment 
tool, we are entertaining yet another concept: the portfolio process has benefits beyond education and 
could be valuable for those already in the field. 

The program portfolio described in this article moves us in the direction of informing practitioners 
who are both knowledgeable and capable of the artistry needed for demands of the day-to-day prac- 
tice. As the demands of the job increase and the pace accelerates, practitioners find fewer opportunities 
to reflect on their practice. Most in-service education involves either learning more of the same kind of 
techniques or making already known techniques more efficient. This has limited potential for profes- 
sional growth because it keeps school psychologists within old, often implicit, schemas. These schemas 
can become quite shop worn and lead to dead-end professional comers. Supervision, evaluation, and 
professional development programs must make reflection an essential component and involve “build- 
ing new understandings to inform our actions in the situation that is unfolding” (Smith, 2001). Cur- 
rently there arc efforts under way to establish a reflective process of supervision and professional 
development in a large, urban school district with over 50 psychologists. This process involves a self- 
assessment of three components: (a) opportunities present to participate in a particular NASP domain, 
(b) the importance of these activities given the personal interests of practicing psychologists and the 
needs of their assigned schools, and (c) current competency or level of preparation to pursue opportu- 
nities related to that domain. Once this reflective process is completed, practitioners create a plan for 
professional development that focuses on creating more opportunities and developing competence in 
chosen areas. All graduate programs in school psychology provide their students with adequate knowl- 
edge and techniques. This is, of course, necessary but in and of itself does not prepare psychologists to 
handle the complexities and ambiguities of professional practice. It is important that school psycholo- 
gists reflect on their practice in the moment and over the long term so that they can creatively meet the 
needs of the children, parents, and educators they serve. 
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